The present invention relates to an image sensing system configured with an image sensing apparatus and a control apparatus, a method of controlling an image sensing system, and a storage medium storing the method.
As conventional video cameras, on the market, which record an image as television signals, there are analog video cameras (e.g., HI-8 type, VHS type) and digital video cameras (e.g., DV type).
In these video cameras, a charge coupled device (CCD) is often used as an image sensing device. Referring to FIG. 23A, in the conventional video cameras, in order to output a moving image as television signals, the CCD is scanned every other scan line as shown by interlace scanning lines 602 to form a first field image 601, then the skipped lines are scanned as shown by interlace scan lines 604 to form a second field image 602. The first and second field images 601 and 602 together form a frame image (interlace scanning).
In the aforesaid interlace scanning method, there is a problem in which, when outputting a frame (configured with two field images) of a fast moving image as a still image, the outputted still image is mismatched because of a difference between the two field images due to the movement associated with a lapse of time. Thus, as a video camera capable of overcoming the above problem, one which sequentially scans all the scan lines 606 to form a frame image 605 as shown in FIG. 23B has recently been on the market (progressive scanning).
Currently, among electronic flashes provided on the aforesaid video cameras, those provided on video cameras performing progressive scanning flash once for each exposure operation for each frame at timing shown in FIG. 5.
Further, in a conventional video camera which performs interlace scanning and outputs two field images as a still image of a frame, there is a problem in which, with only one flashing operation in each frame, the outputted still image is disturbed since one field image is obtained while flashing the electronic flash, whereas the other filed image is obtained without flashing the electronic flash, as shown in FIG. 6, and bright lines and dark lines are mixed every other line.
Meantime, there is a technique for sensing an image with special effects by optically transforming an image by parallel-shifting and/or tilting the optical axis of an image sensing lens. This technique is conventionally known as camera movements. As the camera movements, the parallel translation of a lens in the horizontal direction with respect to the optical axis of the lens is called shift, the parallel translation in the vertical direction is called rise and fall, and the tilting operations of the lens about an axis which is orthogonal to the optical axis in the horizontal and vertical directions are called swing and tilt, respectively.
Further, an image sensed by a video camera is often transferred to a control apparatus, such as a personal computer, via communication means. Furthermore, there are systems in which an image sensing operation of a video camera is controlled by a personal computer from a remote location.
As an example of such systems, a video camera recorder is connected to a personal computer via a cable, e.g., conforming to IEEE 1394 standard and universal serial bus (USB), and while watching an image sensed by the video camera recorder and displayed on the personal computer, a user controls the operation, such as start and stop of image recording, playback of a sensed image, of the video camera recorder from the personal computer with a command transaction set (CTS), further, edits the sensed image on the personal computer.
For interchanging image information and other information between a video camera and a personal computer, the IEEE 1394 standard is prescribed as described above. According to the IEEE 1394 standard, when transferring information, such as image information from a video camera, which requires a fixed transfer rate, isochronous transference is performed, and when transferring other data, asynchronous transference is performed and data is transferred by packet. As a communication protocol, conforming to the IEEE 1394 standard, which determines a format and sequence of information for controlling a video camera, there is an audio video control (AVC) protocol.
Below, a conventional system in which a digital video camera is remote-controlled by a personal computer is explained with reference to the block diagram shown in FIG. 24.
In FIG. 24, reference numeral 1 denotes a digital video camera; 2, a cable conforming to the IEEE 1394 standard; and 3, a personal computer. The CTS is explained, first. In the IEEE 1394 standard, the physical layer, the link layer, and the transaction layer are prescribed by the “IEEE 1394-1995Std”, and a function control protocol (FCP) for controlling devices is prescribed by the “IEC61883CDV” as an upper layer.
In the IEEE 1394 standard, a device controlling another device is called “controller”, and a device controlled by another device is called “target”. In FIG. 24, the personal computer 3 is the controller, and the digital video camera 1 is the target.
As shown in FIG. 25, the controller writes a command to the target by performing write transaction in asynchronous communication according to the IEEE 1394 standard, and the target sends back an acknowledge. Thereafter, the target writes a response to the command to the controller in the similar manner, and the controller also sends back an acknowledge. In the FCP, a command group, called CTS, which is prescribed by the AVC protocol is used in a general-purpose digital video camera recorder, such as the digital video camera 1.
The command group, CTS, defines a sub-unit of the video camera recorder, and control commands, such as REC, PLAY, and STOP, are prepared. These commands are transmitted from the personal computer to the digital video camera 1, which is controlled in accordance with the transmitted commands.
Further, as the CTS, a status command for notifying the computer 3 of a state of the video camera 1 is also prepared so that the computer 3 can be made aware of the current mechanical conditions, such as a time code, of the video camera 1.
In addition, a notify command for inquiring a change in operation condition of the video camera 1 is also prepared as the CTS.
For sensing an image with camera movements by a conventional image sensing system as described above, a user manually controls a camera while checking the special effects on a transformed image caused by the camera movements.
In other words, it is not possible to control the movement of the lens of a video camera from a personal computer to acquire camera movement effects in a conventional image sensing system.
Therefore, for inputting an image with camera movements into the personal computer, it is necessary for a user to manually move the lens of the video camera, which is a troublesome operation.
Further, for confirming the effects of the camera movements on the personal computer, it is necessary to input an image sensed after moving the lens to the personal computer and display it on a display device; thus, a sequence of moving the lens, sensing an image, displaying the image, and confirming the image, has to be repeated a plurality of times. Furthermore, since the camera movements are manually controlled by the user, the amount of movement of the lens is not known by the personal computer.
Furthermore, if lens movement is automated and a system in which the lens is controlled by the personal computer is made, communication for transferring an image and communication for controlling the lens movement have to be performed via different means. Therefore, communication means for communication, such as a cable, needs to be provided independently, which increases the size of an apparatus and the apparatus may not be operated easily.
Further, there is a case of switching between a function of converting visual light into image signals and a function of converting infrared light into image signals, where both functions are provided in the digital video camera 1. However, these functions are not electronically controlled; therefore, the user has to directly operate a switch, provided on the digital video camera 1, for switching between the functions.
Below, the switching between the function of converting visual light into image signals and the function of converting infrared light into image signals in the digital video camera 1 is explained.
FIG. 26 is a block diagram illustrating a brief internal configuration of the digital video camera 1. In FIG. 26, reference numeral 31 denotes a lens; 32, an infrared filter for cutting infrared light; 33, a CCD; 34, a matrix operation circuit; 35, a recorder signal processing circuit; 36, a recorder; 37, a digital interface (DIF); 38, a microcomputer; 39, an actuator; and 40, a switch.
First, in a case of converting visual light into image signals, light passing through the lens 31 is filtered by the infrared filter 32 and the filtered visual light forms an image on the CCD 33. The CCD 33 converts the formed optical image into electric signals, and transmits the signals to the matrix operation circuit 34.
A complementary color filter is formed on the CCD 33 so that each color of the complementary color filter covers each pixels of the CCD, and light of different colors passed through the color filter are incidence on the respective pixels. The signals obtained from the respective pixels are operated in the matrix operation circuit 34, from which a luminance signal Y, and color difference signals R-Y and B-Y are outputted.
The luminance signal Y and the color difference signals R-Y and B-Y are inputted to the recorder signal processing circuit 35. The recorder signal processing circuit 35 converts the inputted image signals (i.e., the luminance signal Y and the color difference signals R-Y and B-Y) into signals of digital video format and provides the converted signals to the recorder 36 where the signals are recorded on a recording medium. At the same time, the signals of the-digital video format are also outputted from the DIF 37.
Next, in a case of converting infrared light into image signals, the user switches the switch 40 to perform infrared processing. In turn, the microcomputer 38 controls the actuator 39 so that the infrared filter 32 is removed out of a light path to the CCD 33 (moved in the direction shown in an arrow A in FIG. 26). Accordingly, light including infrared light incidents on the CCD 33 and converted into electric signals, then transmitted to the matrix operation circuit 34.
Next, the recorder signal processing circuit 35 is controlled so as to change the color difference signals R-Y and B-Y to express achromatic state, because a color component of infrared light is not visible. The subsequent processes are the same as those for processing image signals of visual light.
In the aforesaid manner, even when an image due to visual light is too dark for recognition of its contents, infrared light is converted into an image so that outlines of objects, at least, are clearly seen if infrared light is emitted.
However, in the aforesaid conventional system, there is no command from the personal computer 3 for switching between the function of converting visual light into image signals and the function of converting infrared light into image signals in the digital video camera 1. Therefore, the user has to manually operate the switch provided on the digital video camera 1.
Further, there is a problem in that, when a dark object in a dark place is sensed, an image obtained in the function of converting visual light into image signals does not clearly express outlines, whereas, an image obtained in the function of converting infrared light into image signals does not express color.
USPTO Backgrounds
USPTO Backgrounds