Conventionally, an I/O connection system of this type is used to connect a plurality of computers and a plurality of I/Os in computer apparatuses or the like to make the plurality of computers share the plurality of I/Os. An example of the conventional I/O connection system is described in reference 1 (PCI-SIG, Multi-Root I/O Virtualization and Sharing Revision 0.7, Jun. 8, 2007, p. 28).
Referring to FIG. 12, the I/O connection system described in reference 1 complies with the standards of PCI express (PCIe), and includes a plurality of computers 1-1 to 1-N, a plurality of I/Os 6-1 to 6-M shared by the computers 1-1 to 1-N, and a multi-root PCI express switch 7 (to be referred to as an MR PCIe switch hereinafter) which connects the computers 1-1 to 1-N and the I/Os 6-1 to 6-M to each other. N and M may be different numbers.
The computers 1-1 to 1-N include CPUs 101-1 to 101-N, memories 103-1 to 103-N, and root complexes 102-1 to 102-N which connect the CPUs 101-1 to 101-N, memories 103-1 to 103-N, and MR PCIe switch 7 to each other, respectively. Traditionally, the root complexes 102-1 to 102-N are often called chip sets.
The MR PCIe switch 7 includes upstream PCI-PCI bridges 701-1 to 701-N which are PCI-PCI bridges connected to the computers 1-1 to 1-N, respectively, downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M which are PCI-PCI bridges connected to the I/Os 6-1 to 6-M, respectively, and an internal bus 702 which connects the upstream PCI-PCI bridges 701-1 to 701-N to the downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M inside the MR PCIe switch 7. In the PCIe, the side of the computers 1-1 to 1-N is called upstream, and the side of the I/Os 6-1 to 6-M is called downstream.
The upstream PCI-PCI bridges 701-1 to 701-N are assigned to the computers 1-1 to 1-N exclusively. For example, the upstream PCI-PCI bridge 701-1 is assigned to the computer 1-1. The upstream PCI-PCI bridges 701-1 to 701-N include configuration registers (CFG REGs) 7011-1 to 7011-N, respectively.
The downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M include bridges which are exclusively assigned to the computers 1-1 to 1-N in correspondence with each of the connected I/Os 6-1 to 6-M. For example, the downstream PCI-PCI bridge 703-1-1 is assigned to the computer 1-1. The TLPs (Transaction Layer Packets) of I/O transactions issued by the computers 1-1 to 1-N are transferred via the downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M assigned to the computers 1-1 to 1-N. When transferring TLPs to the I/Os 6-1 to 6-M, the downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M insert, to the TLPs, tags representing the computers 1-1 to 1-N of TLP issue sources. The downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M also erase, from TLPs received from the I/Os 6-1 to 6-M, tags representing the computers 1-1 to 1-N of the TLP issue sources, and then transfer the TLPs to the upstream PCI-PCI bridges 701-1 to 701-N. The downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M include CFG REGs 7031-1-1 to 7031-N-1 to 7031-N-M, respectively.
The CFG REGs 7011-1 to 7011-N and the CFG REGs 7031-1-1 to 7031-N-1 to 7031-N-M hold information of bus numbers and memory spaces which the computers 1-1 to 1-N allocate to the devices connected on the downstream side of the bridges. These pieces of information are used for TLP transfer. The memory spaces, bus numbers, and other information to be used to transfer TLPs will be referred to altogether as PCIe addresses hereinafter. The CFG REGs 7011-1 to 7011-N and the CFG REGs 7031-1-1 to 7031-N-1 to 7031-N-M hold codes to identify the upstream PCI-PCI bridges 701-1 to 701-N and the downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M. When these codes are read in the configuration cycles of the computers 1-1 to 1-N, the MR PCIe switch 7 is recognized and incorporated in the system.
In the MR PCIe switch 7, TLPs are transferred between the upstream PCI-PCI bridges 701-1 to 701-N and the downstream PCI-PCI bridges 703-1-1 to 703-N-1 to 703-N-M, which are assigned to the same computers 1-1 to 1-N. TLP transfer between bridges assigned to different computers 1-1 to 1-N is prohibited.
The I/Os 6-1 to 6-M are complying with the PCIe standards, and can simultaneously be shared by the plurality of computers 1-1 to 1-N. The I/Os 6-1 to 6-M hold, inside them, resources (not shown) to be individually allocated to the computers 1-1 to 1-N. These resources process I/O transactions of the computers 1-1 to 1-N. TLP transmission/reception can be done between the resources of the I/Os 6-1 to 6-M, which are allocated to the same computers 1-1 to 1-N. TLP transmission/reception is prohibited between the resources of the I/Os 6-1 to 6-M, which are allocated to different computers 1-1 to 1-N. To indicate which one of the computers 1-1 to 1-N has issued a TLP that is being processed by the resource allocated to the computer, each of the I/Os 6-1 to 6-M inserts, to the TLP, a tag to identify one of the computers 1-1 to 1-N, and transmits/receives the TLP to/from the MR PCIe switch 7.
The operation of the conventional I/O connection system will be described next.
Assume that the CPU 101-1 of the computer 1-1 has issued an I/O transaction to the I/O 6-1. The root complex 102-1 generates a TLP and transmits it to the MR PCIe switch 7. The upstream PCI-PCI bridge 701-1 receives the TLP, and confirms based on information held in the CFG REG 7011-1 whether the destination of the TLP is the device connected on the downstream side of the bridge. In this case, since the destination is the downstream device, the upstream PCI-PCI bridge 701-1 transmits the TLP to all the downstream PCI-PCI bridges 703-1-1 to 703-1-M assigned to the computer 1-1.
The downstream PCI-PCI bridges 703-1-1 to 703-1-M receive the TLP, and confirm based on information registered in the CFG REGs 7031-1-1 to 7031-1-M whether the destination of the TLP is a device connected on the downstream side of the bridge. In this case, only the downstream PCI-PCI bridge 703-1-1 has the downstream device. The downstream PCI-PCI bridge 703-1-1 adds, to the TLP, a tag to identify the computer 1-1, and transmits the TLP to the I/O 6-1. The I/O 6-1 receives the TLP, and identifies, based on the tag added to it, the computer 1-1 which has transmitted the TLP. The resource of the I/O 6-1, which is allocated to the computer 1-1, executes requested processing.
On the other hand, when transmitting a TLP from the I/O 6-1 to the computer 1-1, the procedure is reverse to that of the operation of causing the computer 1-1 to issue a TLP to the I/O 6-1. At this time, each of the upstream PCI-PCI bridge 701-1 and the downstream PCI-PCI bridge 703-1-1 receives the TLP from downstream, and transfers it to upstream if the destination of the TLP is not the device connected on the downstream side of the bridge.
USPTO Backgrounds
USPTO Backgrounds