When the real world is augmented by appending and displaying information of any kind in a real space, this is known as “augmented reality” (AR). One known AR technology is, for example, technology using a two-dimensional bar code which is printed on a card, for instance. This technology involves capturing an image of a two-dimensional bar code with a camera, identifying the two-dimensional bar code from the captured image in an image processing device, and displaying three-dimensional image data associated with the two-dimensional bar code on a display device, so as to be superimposed on the position of the two-dimensional bar code. In this technology, it is possible to achieve a visual effect which blends a real space with a virtual space, by means of three-dimensional image data which is displayed in superimposed fashion on the captured image.
However, in conventional technology, when a two-dimensional barcode printed on a card is recognized in a captured image, three-dimensional image data is simply displayed in superimposed fashion at the position of the two-dimensional barcode, on the screen which displays the captured image, and there are limitations on the variations of processing for superimposing information on the real space.
One aspect of the present disclosure is an information processing system. The information processing system includes: an arrangement item having a first marker; an arrangement region providing object which provides an arrangement region for the arrangement item and has a plurality of second markers; and an information processing device. The information processing device includes: an imaging processing unit to generate a captured image by capturing images with an imaging device; a positional relationship judgment unit to judge a positional relationship between the first marker of the arrangement item and at least one of the second markers, from a captured image generated by the imaging processing unit which includes the first marker of the arrangement item and at least one of the plurality of second markers of the arrangement region providing object; an information superimposition unit to superimpose predetermined information based on the positional relationship, onto the captured image generated by the imaging processing unit; and a display processing unit to display the captured image on which the predetermined information is superimposed, on a display device. The arrangement item in an information processing system which is one aspect of the present disclosure includes various objects, such as a card, a rectangular solid, a round bar, and the like. The arrangement region providing object in the information processing system which is one aspect of the present disclosure is a sheet, a board, or the like. Furthermore, the arrangement region providing object may be the floor.
According to an information processing system which is one aspect of the present disclosure, information corresponding to a positional relationship between a first marker and a second marker in a real space is superimposed on a captured image generated by the imaging processing unit. By this means, it is possible to superimpose various information on a captured image obtained by capturing images of a real space, in accordance with the positional relationship between the first marker and the second marker in the real space. By diversifying the information superimposed on the captured image obtained by capturing the real space, it is possible to increase the diversity of the augmented real space. Furthermore, since the superimposed information is changed in accordance with the positional relationship between the first marker and the second marker in the real space, then the display of the captured image on which the information is superimposed is made more interesting. Moreover, since the information superimposed on the captured image is set in accordance with the positional relationship between the first marker and the second marker in the real space, then it is possible to superimpose information which is suited to the use of the information processing system, and the usability is improved.
In an information processing system which is one aspect of the present disclosure, a first marker and a plurality of second markers respectively have a shape which enables detection of a position of the marker and an attitude of the marker at the position. Furthermore, the positional relationship judgment unit of the information processing device may judge one or a combination of the distance between the first marker and the second marker, the orientation of the first marker with respect to the second marker, and the direction of the position of the first marker with respect to the second marker, as the positional relationship between the first marker and the second marker, based on the shape of the first marker or the shape of at least one second marker which is included in the captured image. The information processing system judges various positional relationships between the first marker and one of the plurality of second markers in the real space, and the information which is superimposed on the captured image is diversified in accordance with the judged positional relationship.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit may superimpose predetermined information at a position on the arrangement region providing object on the captured image which is based on a positional relationship between the first marker and at least one second marker by the positional relationship judgment unit. By this means, in addition to being able to set the content of superimposed information on the basis of the positional relation based on ship between the first marker and at least one second marker, it is also possible to set the position at which the information is superimposed, and greater variety is achieved in the display of the captured image on which information is superimposed.
In an information processing system which is one aspect of the present disclosure, arrangement of the arrangement item on the arrangement region providing object is changed by a user; the imaging processing unit may generate a captured image by capturing images successively at predetermined intervals by the imaging device; the positional relationship judgment unit of the information processing device may detect change in the positional relationship between the first marker and at least one second marker which is included in the captured image, due to the change in the arrangement of the arrangement item, from captured images which are generated at predetermined intervals; and the information superimposition unit may superimpose the predetermined information on the captured image which is generated by the imaging processing unit, in accordance with the change in the positional relationship. In the real space, the position of the arrangement item on the arrangement region providing object is changed by the user, and predetermined information is superimposed and displayed on a captured image obtained by capturing the real space, in accordance with the change in the positional relationship between the first marker and the second marker resulting from this change in the position of the arrangement item, and therefore it is possible to provide a user with a feeling of an augmented real space. Furthermore, since the real space is augmented by superimposing information on a captured image obtained by capturing the real space, in accordance with change in the positional relationship between the first marker and a second marker in the real space, then greater interest is created.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose predetermined information on a captured image generated by the imaging processing unit, if change in the distance between the first marker and the second marker which are included in the captured image is detected as change in the positional relationship between the first marker and the second marker. By this means, it is possible to vary the information that is superimposed on the captured image obtained by capturing the real space, in accordance with change in the distance between the first marker and the second marker, and the display of the captured image on which information is superimposed is made more diverse.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose predetermined information indicating a passing of time from previous superimposition of information as predetermined information, when change in the distance between the first marker and the second marker which are included in the captured image is detected as change in the positional relationship between the first marker and the second marker. Change in a numerical value such as change in the distance between the first marker and the second marker in the real space is reflected in a temporal element, such as passing of time in the information superimposed on the captured image, and the variety of the effects which are applied to the augmented real space are increased by superimposing information on the captured image based on change in the real space.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose predetermined information on a captured image, when change in the orientation of the first marker with respect to the second marker which are included in the captured image is detected as change in the positional relationship between the first marker and the second marker. By this means, it is possible to set the information to be superimposed on the captured image in accordance with change in the orientation of the first marker with respect to the second marker, and the diversity of the real space which is augmented by the superimposition of information is raised.
In an information processing system which is one aspect of the present disclosure, at least one second marker included in the captured image may define a position on an arrangement region and a direction, with reference to itself, based on its shape. Moreover, the information superimposition unit of the information processing device may superimpose predetermined information on a captured image generated by the imaging processing unit, when change in the direction in which the first marker is positioned with reference to the second marker is detected as change in the positional relationship between the first marker and the second marker which are included in the captured image. By this means, it is possible to vary the information that is superimposed on the captured image, in accordance with change in the direction in which the first marker is positioned with reference to the second marker, and hence the diversity of the real space which is augmented by the superimposition of information is increased.
In an information processing system which is one aspect of the present disclosure, the arrangement region providing object may have a plurality of divided regions of a predetermined size inside the arrangement region, and a plurality of second markers may be allocated respectively to a portion of the plurality of divided regions. Moreover, the information superimposition unit of the information processing device may superimpose the predetermined information based on a divided region to which the at least one second marker is allocated. Consequently, it is possible to superimpose predetermined information on the captured image based on the divided regions of the arrangement region providing object.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose predetermined information on a captured image generated by the imaging processing unit when the first marker is arranged on a divided region which is allocated to at least one of the second markers included in the captured image. When the user has arranged the first marker on a divided region allocated to a second marker in the real space, information is superimposed on the captured image and therefore increased usefulness and interest is created by the display of the captured image on which information is superimposed.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose the predetermined information, when the positional relationship judgment unit judges that the first marker is arranged in a range including a predetermined number of divided regions about the periphery of the divided region allocated to the at least one second marker included in the captured image. By this means, when the first marker is arranged in a range including a divided region which is allocated to a second marker, predetermined information is superimposed on the captured image, and hence greater diversity is achieved in the real space which is augmented by the superimposition of information.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose the predetermined information on the captured image, when the first marker is removed from a range including a predetermined number of divided regions about the periphery of the divided region allocated to the at least one second marker included in the captured image. By this means, when the first marker is removed from a range including a divided region which is allocated to a second marker, predetermined information is superimposed on the captured image, and hence greater diversity is achieved in the real space which is augmented by the superimposition of information.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit of the information processing device may superimpose information which induces passing of time of the information superimposed on the captured image, when change in the divided region where the first marker is positioned is detected. In this way, it is possible to reflect change in the position of the first marker in the real space as change in the time of the augmented real space by superimposing information on the captured image, and greater variety is achieved in the effects applied to the augmented real space in accordance with change in the real space.
An information processing system which is one aspect of the present disclosure may further include a divided region judgment unit to judge a positional relationship between a first marker and a second marker based on the arrangement of a plurality of divided regions, from the positional relationship judged by the positional relationship judgment unit included in the information processing device. By this means, it is possible to judge the positional relationship between the first marker and the second marker in units of the divided regions, and the judgment of the positional relationship is carried out easily.
In an information processing system which is one aspect of the present disclosure, the information superimposition unit included in the information processing device may superimpose predetermined information on a captured image in accordance with the positional relationship between the first marker and a second marker located closest to the first marker, when the captured image includes the first marker and a plurality of second markers. By this means, even in cases where there are a plurality of markers about the periphery of a first marker, the information processing device simply carries out processing for judging the positional relationship with respect to the nearest second marker, and hence the processing load is reduced.
In the information processing system which is one aspect of the present disclosure, the first marker and each of the plurality of second markers may respectively be indicators of display positions of objects superimposed on the real space captured by the imaging device. Furthermore, the display processing unit of the information processing device may cause a display device to display respectively corresponding objects after implementing superimposition at positions corresponding to the first marker and the plurality of second markers which are included in the captured image generated by the imaging processing unit. The real space is augmented by superimposing and displaying objects corresponding to the respective markers at positions corresponding to each marker, on the captured image.
In an information processing system which is one aspect of the present disclosure, the arrangement item may be a card, the arrangement region providing object may be a board, and the information processing device may further have a game development control unit to control development of a game in which the status is changed due to the user arranging a card on the board. In this way, when the information processing system provides a game, then the game is developed by arranging a card on the board, and various information is superimposed on a captured image displayed on a display device of the information processing device, in accordance with the development of the game, thereby making it possible to provide a highly interesting game.
Moreover, the aspect of the present disclosure may be a method, system, information processing device or a recording medium recoding a program which is read by a computer or other device or machine. Here, the recording medium which is read by the computer, or the like, is a recording medium on which information such as data or a program is stored by an electrical, magnetic, optical, mechanical or chemical action, and from which information is read by a computer, or the like.
For example, one aspect of the present disclosure is an information processing method which is executed by an information processing device in an information processing system, including: an arrangement item having a first marker; an arrangement region providing object which provides an arrangement region for the arrangement item and has a plurality of second markers; and an information processing device which is connected to an imaging device. In this information processing method, the information processing device executes: generating a captured image by capturing images with an imaging device; judging a positional relationship between the first marker and at least one second marker, from the generated captured image, the image being configured to include the first marker of the arrangement item and at least one of the plurality of second markers of the arrangement region providing object; superimposing predetermined information based on the positional relationship, onto the generated captured image; and causing a display device to display a captured image on which the predetermined information is superimposed.
Moreover, for example, one aspect of the present disclosure is an information processing device, including: an imaging processing unit to generate a captured image by capturing images with an imaging device; a positional relationship judgment unit to judge a positional relationship between a first marker provided on an arrangement item and at least one of a plurality of second markers provided on an arrangement region providing object which provides an arrangement region to the arrangement item, from a captured image generated by the imaging processing unit which includes the first marker and the at least one second marker; an information superimposition unit to superimpose predetermined information based on the positional relationship, onto a captured image generated by the imaging processing unit; and a display processing unit to cause a display device to display a captured image on which the predetermined information is superimposed.
Moreover, for example, one aspect of the present disclosure is a recording medium recording an information processing program executed by a computer of an information processing device which carries out imaging by an imaging device, the program causing the computer to function as: an imaging processing unit to generate a captured image by capturing images with an imaging device; a positional relationship judgment unit to judge a positional relationship between a first marker provided on an arrangement item and at least one of a plurality of second markers provided on an arrangement region providing object which provides an arrangement region to the arrangement item, from a captured image generated by the imaging processing unit which includes the first marker and the at least one second marker; an information superimposition unit to superimpose predetermined information based on the positional relationship, onto a captured image generated by the imaging processing unit; and a display processing unit to cause a display device to display a captured image on which the predetermined information is superimposed.
USPTO Backgrounds
USPTO Backgrounds