close

Вход

Забыли?

вход по аккаунту

?

JP2016119621

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2016119621
Abstract: In a case where an omnidirectional camera and a microphone array are integrally used,
a microphone such that the reference direction of the imaging direction of the omnidirectional
camera and the reference direction of the sound collection direction of the microphone array
match with easy operation. Adjust the reference direction of sound collection direction of the
array. SOLUTION: A part of a casing of an omnidirectional camera is opened to capture a marker
61, and a 360-degree captured image 99 including the marker 61 captured by the
omnidirectional camera and a camera reference heading radially from an image center 101 The
line 95 is displayed, and by designating the position of the marker 61 of the captured image 99,
the microphone reference line 97 heading in the radial direction passing through the designated
position and the image center 101 is displayed superimposed on the captured image 99. The
angle formed between the line 95 and the microphone reference line 97 is calculated as a shift
amount ω, and the horizontal angle in the sound collection direction of the microphone array is
adjusted using this shift amount ω. [Selected figure] Figure 6
Calibration method
[0001]
The present invention relates to a calibration method for adjusting a deviation between a
omnidirectional camera having an imaging range of 360 degrees and a microphone array having
a sound collection range of 360 degrees from a reference direction of each horizontal angle.
[0002]
By combining and using an omnidirectional camera with a 360-degree imaging range and a
microphone array that picks up sound from the direction of 360-degrees, it is possible to further
03-05-2019
1
optimize the display of images desired by the user and the detection of sound. Technology has
been proposed.
[0003]
For example, Patent Document 1 discloses a system for an omnidirectional camera and a
microphone array that determines the direction of sound producing an audio signal.
According to this technique, the portion of the image / video that includes the speaker can be
identified and associated with the audio signal so that the view of the camera seen in the video
conference can be directed when the speaker speaks .
[0004]
Further, Patent Document 2 discloses a voice processing system and a voice processing method
capable of generating voice data in which voice data in an arbitrary direction is emphasized.
According to this technology, when a designated part of voice emphasis is designated on the
screen during playback or pause of a video, audio data in a direction from the microphone array
toward a position (sound source) corresponding to the designated part Is emphasized.
[0005]
In the above-described prior art, the microphone array has an annular shape in which a plurality
of microphones are arranged in the circumferential direction. The omnidirectional camera is
coaxially disposed in the inner hole of the microphone array substantially on the same plane. If
the omnidirectional camera and the microphone array are mounted parallel to a horizontal
surface such as a ceiling surface, for example, an angle in a rotational direction about the
common central axis perpendicular to the same plane can be referred to as a horizontal angle. .
The omnidirectional camera and the microphone array are used in combination to optimize
image display and audio detection (for example, when emphasizing audio data corresponding to
a designated portion on the image), the center described above It is necessary that the respective
reference direction (for example, 0 degree direction) positions of the angles (horizontal angles) in
03-05-2019
2
the rotational direction about the axis coincide with each other. In the case where an
omnidirectional camera and a microphone array arranged coaxially are used in combination, the
included angle and elevation angle (vertical angle) with the horizontal plane passing through the
omnidirectional camera as viewed from the omnidirectional camera and the microphone array
are the same. It is a value.
[0006]
JP 2004-322782 JP 2014-143678
[0007]
However, the omnidirectional camera requires an angle of view (that is, a lens) that defines the
range of the captured image so that the passage becomes straight, from the request of the user
(for example, the observer) to improve the visibility depending on the subject such as the
passage. You may want to rotate and adjust.
For example, when initially installing the omnidirectional camera and the microphone array, or
when installing the omnidirectional camera and the microphone array after integrally removing
the existing omnidirectional camera. When the adjustment of the angle of view of the captured
image is performed based on the request for improvement of the user's visibility described above,
the horizontal angles of the omnidirectional camera and the microphone array are respectively
set. The reference direction of (for example, the 0 degree direction) does not match. As a result, a
situation occurs in which the audio data corresponding to the desired designated part on the
image can not be emphasized.
[0008]
If the user views the image (captured image) captured by the omnidirectional camera while
observing the situation described above, the actual sound collection direction of the microphone
array is measured, and the reference position of each horizontal angle is calibrated, It will require
a great deal of effort and time.
[0009]
The present invention is devised in view of the above-mentioned conventional situation, and
when the omnidirectional camera and the microphone array are integrally used, the reference
direction of the imaging direction of the omnidirectional camera and the sound collection of the
03-05-2019
3
microphone array can be easily performed. It is an object of the present invention to provide a
calibration method of adjusting the reference direction of the sound collection direction of the
microphone array so that the reference direction of the direction matches.
[0010]
The present invention is a calibration method for adjusting the sound collection direction of a
microphone array that is coaxially attached surrounding an omnidirectional camera capable of
capturing an omnidirectional and capable of collecting omnidirectional sound, which is all the
above-mentioned calibration methods. A marker imaging step of imaging the marker provided in
the part with the omnidirectional camera in response to opening of a part of the casing of the
azimuth camera, and omnidirectional imaging including the marker imaged by the
omnidirectional camera A camera reference line display step of displaying on a display unit an
image and a camera reference line indicating a reference direction of an imaging direction of the
omnidirectional camera; and designation of the marker in the captured image displayed on the
display unit. A microphone reference line display step of displaying a microphone reference line
indicating a reference direction of the sound collection direction of the microphone array so as to
be superimposed on the captured image; the camera reference line and the microphone
reference line The angle calculated as an adjustment angle, using the adjustment angle, having an
adjustment step of adjusting the horizontal angle of the sound collection direction of the
microphone array, a calibration method.
[0011]
According to the present invention, when the omnidirectional camera and the microphone array
are integrally used, the reference direction of the imaging direction of the omnidirectional
camera and the reference direction of the sound collecting direction of the microphone array
coincide with each other by easy operation. You can adjust the reference direction of the sound
collection direction of the microphone array.
[0012]
A conceptual diagram of an image pickup and sound collection system that executes the
calibration method of the present embodiment (A) a perspective view of an image pickup and
sound collection unit shown in FIG. 1, (B) an image pickup and sound pickup unit when an
omnidirectional camera is retrofitted An exploded perspective view A sectional view of an image
pickup unit attached to a ceiling by a mounting member A perspective view of an image pickup
unit with the sub cover of an omnidirectional camera opened A block diagram of an image
pickup system A schematic view showing the setup screen at the time, (B) A schematic view
showing the setup screen when the marker is clicked, (C) A schematic view showing the setup
screen when the marker position is determined Omnidirectional camera at the calibration setup
03-05-2019
4
Flow chart explaining the processing procedure of PC Flow chart explaining the processing
procedure of PC at the time of calibration setting
[0013]
Hereinafter, an embodiment (hereinafter, referred to as “the present embodiment”) which
specifically discloses the calibration method according to the present invention will be described
in detail with reference to the drawings.
In the calibration method of the present embodiment, an omnidirectional camera capable of
capturing an omnidirectional (that is, 360 degrees) image and a microphone array capable of
collecting omnidirectional (that is, 360 degrees) sounds are integrated. When used, the reference
direction of the horizontal angle of the microphone array's sound collection direction so that the
reference direction of the horizontal angle of the omnidirectional camera's imaging direction
matches the reference direction of the horizontal angle of the microphone array's sound
collection direction. Be adjustable.
[0014]
FIG. 1 is a conceptual view of an imaging and sound pickup system that executes the calibration
method of the present embodiment.
The calibration method of the present embodiment can be suitably used, for example, for the
image pickup system 11 configured by an omnidirectional camera and a microphone array.
The imaging and sound pickup system 11 can be installed, for example, in a factory, a store (for
example, a retail store, a bank), a meeting room, or a public place (for example, a station, a
library, an event site).
The imaging and sound collecting system 11 shown in FIG. 1 is configured to include an
omnidirectional camera 13 and a microphone array 15, a POE hub 17, a network 19, and a PC
21 which are used together as one unit.
03-05-2019
5
The omnidirectional camera 13 and the microphone array 15 each have an IP address, and are
connected to the POE hub 17 via the two cables 23 and 25.
[0015]
FIG. 2A is a perspective view of the imaging and sound pickup unit shown in FIG. FIG. 2B is an
exploded perspective view of the image pickup unit when the omnidirectional camera is
retrofitted. As shown in FIG. 2A, the omnidirectional camera 13 and the microphone array 15 are
products (one-piece product) in which separate omnidirectional cameras 13 and the microphone
array 15 are separately set and shipped. As shown to B), it may be used as a combination product
which combines the single product of only omnidirectional camera 13 or only microphone array
15 integrally. The calibration method of the present embodiment can be applied to any of these
integral products and combination products. In the case of an integrated product, the
omnidirectional camera 13 and the microphone array 15 are mounted by fitting the
omnidirectional camera 13 into the inner hole 27 of the annular microphone array 15, as shown
in FIG. 2A. In the case of a combination product, as shown in FIG. 2B, the cover 29 for closing the
inner hole 27 of the microphone array 15 is removed, and then the omnidirectional camera 13 is
attached to the inner hole 27 so as to be integrated. Become.
[0016]
The omnidirectional camera 13 is provided with, for example, a fisheye lens, and can obtain a
captured image of omnidirectional (that is, 360 degrees). In addition, the microphone array 15
has a housing in which a plurality of microphones are arranged at equal intervals in the
circumferential direction, and can collect sounds in all directions (that is, 360 degrees). The
omnidirectional camera 13 and the microphone array 15 are mounted with their central axes
coincident with each other in any case of the integrated product and the combination product
described above. That is, the microphone array 15 has an annular shape in which a plurality of
microphones are arranged in the circumferential direction centering on the same axis. The
omnidirectional camera 13 is coaxially disposed in the inner hole 27 of the microphone array 15
on substantially the same plane. For example, when the omnidirectional camera 13 and the
microphone array 15 are mounted in parallel to a horizontal surface such as a ceiling surface, the
angle of the rotational direction centered on the common central axis (coaxial) is a horizontal
angle. It becomes. In addition, the omnidirectional camera 13 and the microphone array 15 are
coaxially arranged when they are integrally used as in this embodiment, so that the vertical angle
with the horizontal plane passing through the omnidirectional camera 13 (in other words, The
03-05-2019
6
vertical angle in the imaging direction as viewed from the omnidirectional camera 13 and the
vertical angle in the sound collecting direction as viewed from the microphone array 15 are
treated as the same.
[0017]
FIG. 3 is a cross-sectional view of an imaging and sound pickup unit attached to a ceiling by an
attachment member. The mounting member 33 is attached and fixed to the installation surface
(for example, the ceiling 31) of the omnidirectional camera 13 and the microphone array 15, and
both the omnidirectional camera 13 and the microphone array 15 are attached to the mounting
member 33. Thereby, the omnidirectional camera 13 and the microphone array 15 are
integrated and united. The attachment member 33 is a metal member formed in a substantially
disk shape having irregularities on the surface. The mounting member 33 may be a member
made of ceramic or synthetic resin (for example, plastic or elastomer).
[0018]
On the surface of the mounting member 33, that is, the surface of the mounting member 33
facing the ceiling surface, there are concentrically three camera locking pieces 35 projecting
coaxially and for mounting and fixing the omnidirectional camera 13. It is formed. Furthermore,
on the surface of the mounting member 33, the microphone locking piece 37 projecting in the
coaxial direction and for mounting and fixing the microphone array 15 is concentrically larger in
diameter than the concentric circle on which the camera locking piece 35 is formed. It is formed
in three places.
[0019]
An engagement hole 41 engaged with a camera fixing pin 39 provided on the bottom surface of
the omnidirectional camera 13 in the camera locking piece 35 is substantially in a gourd shape
whose diameter at one end is larger than that at the other end. It is formed. Similarly, in the
microphone locking piece 37, the engagement hole 45 engaged with the microphone fixing pin
43 provided on the bottom surface of the microphone array 15 has a diameter substantially
larger than that of the other end. It is formed in the shape of a circle. The camera fixing pin 39
and the microphone fixing pin 43 have a head (having a diameter (diameter) between one end
and the other end of the engaging hole 41 and the engaging hole 45 respectively and a trunk
03-05-2019
7
thinner than the head Become.
[0020]
When the omnidirectional camera 13 and the microphone array 15 are attached to the
attachment member 33, first, the omnidirectional camera 13 is attached to the attachment
member 33. In this case, the camera fixing pin 39 is engaged with the engagement hole 41
formed in the camera locking piece 35. Then, the camera fixing pin 39 is moved in the
engagement hole 41 by rotating the omnidirectional camera 13 in a state where the head of the
camera fixing pin 39 protrudes from the engagement hole 41. Then, with the head of the camera
fixing pin 39 moved to the other end side of the engaging hole 41, the camera fixing pin 39 and
the engaging hole 41 are engaged, and the omnidirectional camera 13 is coaxial. It is fixed in the
direction.
[0021]
After the omnidirectional camera 13 is attached to the attaching member 33, the microphone
array 15 is attached to the attaching member 33 so that the omnidirectional camera 13 is
exposed from the inner hole 27 of the microphone array 15. In this case, the microphone fixing
pin 43 is engaged with the engagement hole 45 formed in the microphone locking piece 37. The
procedure for fixing the microphone fixing pin 43 to the engagement hole 45 is the same as the
procedure for fixing the camera fixing pin 39 to the engagement hole 41.
[0022]
As described above, the omnidirectional camera 13 and the microphone array 15 share the
central axis 47 with the attachment member 33 attached and fixed to the ceiling surface, and the
horizontal direction of the imaging direction of the omnidirectional camera 13 and the collection
of the microphone array 15 It is integrally attached in the state where each reference directions
(for example, direction which shows 0 degree) of the horizontal direction of a sound direction
correspond. The integrated omnidirectional camera 13 and the microphone array 15 can be
referred to as an imaging and sound pickup unit 49.
[0023]
03-05-2019
8
However, here, in order to improve the visibility of the image captured by the omnidirectional
camera 13 of the user, the adjustment of the angle of view of the captured image is performed by
adjusting the adjustment screw 59 exposed according to the opening of the sub cover 57
described later. It is assumed that the reference directions (for example, the 0 degree direction)
of the horizontal angles of the omnidirectional camera and the microphone array no longer
coincide with each other (see FIG. 4). For example, as shown in FIG. 6A, the adjustment of the
angle of view of the captured image is adjusted so that the passage becomes straight in the
vertical direction, and the reference direction (eg, 0) of each horizontal angle of the
omnidirectional camera and the microphone array It is assumed that the direction is not
identical.
[0024]
FIG. 4 is a perspective view of the image pickup unit with the sub cover of the omnidirectional
camera opened. In the omnidirectional camera 13, the housing 51 is formed in a flat cylindrical
shape having a hole 53 at the top of the conical portion. A dome cover 55 protruding from the
hole 53 to the outside of the case is attached to the case 51. The dome cover 55 covers the lens
unit (not shown) on the inner side. The lens unit is rotatable relative to the housing 51 by an
adjustment gear unit (not shown). The lens unit is rotated integrally with an imaging device (not
shown). That is, the omnidirectional camera 13 can rotate (image rotation) the captured image by
rotation of the lens unit.
[0025]
In the omnidirectional camera 13, a sub cover 57 which is a part of the housing 51 is attached so
as to be able to open and close. The sub cover 57 is rotatably attached via a hinge shaft, and its
movement is restricted except in the rotational direction about the hinge shaft. That is, the sub
cover 57 is opened and closed so that it does not go out of the locus in the rotational direction
and rattling occurs. Therefore, the sub cover 57 opened to the open end is not displaced.
Preferably, the sub cover 57 is releasably held by the moderation mechanism at the position of
the open end.
[0026]
03-05-2019
9
In the omnidirectional camera 13, the adjustment screw 59 is exposed to the inside of the
housing 51 by opening the sub cover 57. The adjusting screw 59 drives the adjusting gear unit.
That is, the omnidirectional camera 13 can adjust the rotation of the lens unit via the adjustment
gear unit by opening the sub cover 57 and rotating the adjustment screw 59 with a tool. As
described above, the omnidirectional camera 13 and the microphone array 15 are fixed to the
ceiling 31 by the attachment member 33. The relative rotation of the fixed omnidirectional
camera 13 and the microphone array 15 is restricted. In this state, only the lens unit of the
omnidirectional camera 13 is rotated by the adjusting screw 59. Thereby, the omnidirectional
camera 13 is configured such that the user can visually recognize the captured image by rotating
it to a desired angle. However, the reference directions (for example, the 0 degree direction) of
the horizontal angles of the omnidirectional camera and the microphone array do not match.
[0027]
The omnidirectional camera 13 can be configured to be able to detect the amount of rotation of
the adjustment gear unit. That is, the omnidirectional camera 13 is a camera adjustment unit
provided with an adjustment gear unit, a segment gear of a lens unit with which the adjustment
gear unit meshes, or an angle sensor that detects the amount of rotation of the lens unit itself (for
example, a lens barrel) Can be provided. As a result, the omnidirectional camera 13 can always
store the reference direction (for example, the direction in which the horizontal angle indicates 0
degree) of the horizontal angle in the imaging direction as numerical information in the storage
unit.
[0028]
The marker 61 is provided on the back of the sub cover 57. The marker 61 is included in the
imaging range of the omnidirectional camera 13 by opening the sub cover 57. That is, the
marker 61 is projected on the captured image of the omnidirectional camera 13 with the sub
cover 57 opened. The marker 61 represents the reference position of the horizontal angle in the
sound collection direction of the microphone array 15. The sound collection direction is a
direction from the microphone array 15 toward an audio position corresponding to a designated
position on the captured image of the omnidirectional camera 13 displayed on the display unit
73 by the operation of the operation unit 79 of the PC 21 (in other words, The directivity is
formed in the PC 21 and the voice data of the voice collected by the microphone array 15 is
emphasized and output in the voice direction. Once the omnidirectional camera 13 and the
microphone array 15 are fixed by the mounting member 33, they do not rotate relative to each
03-05-2019
10
other. Therefore, the sub cover 57 can be provided with the marker 61 associated with the
reference direction (for example, the direction in which the horizontal angle indicates 0 degree)
of the horizontal angle in the sound collection direction of the microphone array 15.
[0029]
In the imaging and sound collecting system 11, the sub cover 57 is closed during normal
operation, so the marker 61 is not imaged. Of course, during normal operation, the microphone
array 15 is not imaged. As described above, in the imaging and sound pickup system 11 to which
the calibration method of the present embodiment can be applied, the sub cover 57 can be closed
to hide the marker 61 during normal operation, and the omnidirectional camera 13 and the
microphone array 15 can be used. At the time of calibration, the sub cover 57 can be opened to
expose the marker 61 serving as the reference position of the microphone array 15.
[0030]
FIG. 5 is a block diagram of an imaging and sound collecting system. The microphone array 15 is
configured to include a plurality of (for example, three in FIG. 5) microphones 63, a control unit
65, a storage unit 67, and a communication unit 69.
[0031]
A plurality of microphones 63 are arranged along the circumferential direction of the
microphone array 15. For the microphone 63, for example, a high-quality small electret
condenser microphone (ECM: Electret Condenser Microphone) is used.
[0032]
The control unit 65 controls the operation of each unit of the microphone array 15, further
performs A / D conversion of audio data collected by each of the microphones 63, and executes
predetermined audio processing on a digital audio signal. .
[0033]
The storage unit 67 stores the audio signal collected by the microphone array 15.
03-05-2019
11
[0034]
The communication unit 69 transmits the audio signal stored in the storage unit 67 to the PC 21
via the network 19 in response to an instruction from the control unit 65.
[0035]
The PC 21 includes a communication unit 71, a display unit 73, a storage unit 75, an audio
output unit 77, an operation unit 79, and a control unit 81.
[0036]
The communication unit 71 transmits and receives signals between the microphone array 15 and
the omnidirectional camera 13 via the network 19.
[0037]
The display unit 73 displays an imaging signal (that is, data of a captured image) transmitted
from the omnidirectional camera 13, an input operation unit and the like provided in an
operation area used in calibration described later (see FIG. 6).
[0038]
The storage unit 75 indicates the reference direction of the horizontal direction of the operation
direction and the microphone array 15 (that is, the reference direction of the horizontal angle of
the sound collecting direction of the microphone array 15 or the reference direction of the
horizontal angle of the sound collecting direction of the microphone array 15). Drawing data of
line), reference direction of omnidirectional camera 13 (that is, drawing data of camera reference
line showing reference direction of horizontal angle of omnidirectional camera 13 and reference
direction of horizontal angle of omnidirectional camera 13) .
[0039]
The voice output unit 77 is voice data on which the control unit 81 performs emphasis
processing in a specific direction based on voice data collected by the microphone array 15 or
voice data thereof (that is, from the microphone array 15, an operation unit The voice data of the
directivity formed in the direction (sound collecting direction) toward the voice position
corresponding to the position on the screen of the display unit 73 designated by the operation to
79 is voice-outputted.
03-05-2019
12
[0040]
The operation unit 79 is disposed, for example, in correspondence with the screen of the display,
and is configured using a touch panel or a touch pad that allows an input operation with a user's
finger or a stylus pen.
The operation unit 79 may be configured using a pointing device such as a mouse or a keyboard.
[0041]
The control unit 81 is configured using, for example, a central processing unit (CPU), a micro
processing unit (MPU), or a digital signal processor (DSP).
The control unit 81 performs control processing (for example, directivity formation processing)
for overall control of image processing and sound processing operations, data input / output
processing with the omnidirectional camera 13 and the microphone array 15, data Execute
calculation (calculation) processing and data storage processing.
[0042]
The omnidirectional camera 13 is configured to include an imaging unit 83, a camera adjustment
unit 85, a communication unit 87, a storage unit 89, a control unit 91, and a microphone
cooperation unit 93.
[0043]
The imaging unit 83 at least includes a lens unit, and an imaging element that can be configured
using a charge device couples (CCD) or a complementary metal oxide semiconductors (CMOS).
The lens unit is rotated to be integrated with the imaging device.
03-05-2019
13
[0044]
The camera adjustment unit 85 has the above-mentioned adjustment gear unit.
The camera adjustment unit 85 rotates the lens unit via the adjustment gear unit when the
adjustment screw 59 is rotated.
The camera adjustment unit 85 can be provided with an angle sensor that obtains the amount of
rotation of the lens unit as rotation information.
Thereby, the camera adjustment unit 85 can always grasp the reference position of the lens unit.
[0045]
The communication unit 87 transmits an imaging signal (that is, data of a captured image)
captured by the imaging unit 83 to the PC 21 via the network 19 as part of the imaging
information.
Further, the communication unit 87 exchanges data with the PC 21 regarding the reference
direction of the omnidirectional camera 13.
[0046]
The storage unit 89 stores an imaging signal (that is, data of a captured image) captured by the
imaging unit 83.
In addition, the storage unit 89 is a reference direction (adjustment direction) of the horizontal
angle of the current imaging direction of the omnidirectional camera 13 or an omnidirectional
camera at the time of shipment (that is, at the beginning of executing the calibration method of
this embodiment). Information of 13 reference directions (a reference direction of a horizontal
angle of an imaging direction of omnidirectional camera 13 and a reference direction of a camera
03-05-2019
14
reference line indicating a reference direction of a horizontal angle of an imaging direction of
omnidirectional camera 13) is stored.
Information on the reference direction of the omnidirectional camera 13 (see above) is stored in
an overlapping manner with the storage unit 75 of the PC 21.
[0047]
The control unit 91 controls the operation of each part of the omnidirectional camera 13, and
adjusts various parameters related to imaging of the omnidirectional camera 13.
[0048]
The microphone cooperation unit 93 includes information on the reference direction of the
horizontal angle in the sound collection direction of the microphone array 15 and the reference
direction of the horizontal angle of the sound collection direction of the microphone array 15
after adjustment by the calibration method of the present embodiment. Control the
communication with the PC 21 via the communication unit 87.
[0049]
Next, the calibration method of the present embodiment will be described with reference to FIGS.
6 (A) to 6 (C).
FIG. 6A is a schematic view showing a setting screen at the start of display of a captured image.
FIG. 6B is a schematic view showing a setting screen when the marker is clicked. FIG. 6C is a
schematic view showing a setting screen when the position of the marker is determined.
[0050]
The calibration method of the present embodiment includes the steps of exposing the marker 61
(marker exposing step), displaying the camera reference line 95 (camera reference line
displaying step), and displaying the microphone reference line 97 (FIG. And a step of adjusting
the horizontal angle in the sound collection direction of the microphone array 15 (adjustment
step).
03-05-2019
15
[0051]
In the step of exposing the marker 61, the sub cover 57 which is a part of the casing 51 of the
omnidirectional camera 13 is opened (see FIG. 4).
Thereby, the marker 61 provided on the sub cover 57 is exposed.
[0052]
In the step of displaying the camera reference line 95, as shown in FIG. 6A, the PC 21 picks up an
imaged image 99 of omnidirectional (that is, 360 degrees) including the marker 61 imaged by
the omnidirectional camera 13; The display unit 73 displays a camera reference line 95
indicating the reference direction of the imaging direction of the omnidirectional camera 13 in
the radial direction from the image center 101 of the image 99. When the captured image 99 is
displayed in color, the camera reference line 95 is displayed in blue, for example.
[0053]
In the step of displaying the microphone reference line 97, as shown in FIG. 6B, when the
position of the marker 61 displayed in the captured image 99 is specified by the PC 21 in the
order, as shown in FIG. A microphone reference line 97 indicating the reference direction of the
sound collection direction of the microphone array 15 going in the radial direction passing
through the position and the image center 101 is superimposed on the captured image 99 and
displayed.
[0054]
In the calibration method of the present embodiment, the microphone reference line 97 initially
displayed by specifying the position of the marker 61 is displayed by a broken line as shown in
FIG. 6B, for example.
When the captured image 99 is displayed in color, the microphone reference line 97 is displayed
03-05-2019
16
in red, for example.
[0055]
The microphone reference line 97 displayed as a broken line is displayed as a solid line as shown
in FIG. 6C by input of a confirmation instruction by the user (for example, a click on the setting
button 103). Thereby, the PC 21 can hold the value of the difference between the reference
direction of the horizontal angle of the sound collection direction of the microphone array 15
and the reference direction of the horizontal angle of the imaging direction of the
omnidirectional camera 13.
[0056]
In the calibration method of the present embodiment, when the microphone reference line 97 is
determined, the camera reference line 95 moves to the microphone reference line 97 (that is, the
camera reference line 95 matches the microphone reference line 97). Instead, the camera
reference line 95 and the microphone reference line 97 remain as they are. Therefore, the value
of the difference between the reference direction of the horizontal angle of the sound collection
direction of the microphone array 15 and the reference direction of the horizontal angle of the
imaging direction of the omnidirectional camera 13 is held as a difference parameter in the PC
21 or the omnidirectional camera 13 When the user designates the position on the captured
image of the omnidirectional camera 13 displayed on the display unit 73 using the operation unit
79, the directivity is formed at the sound position corresponding to the designated position. The
direction of directivity (orientation direction) is calculated in consideration of the parameters.
[0057]
In addition to the input of the position of the marker 61 on the captured image described above,
the microphone reference line 97 is also displayed by numerically inputting an angle by the
operation unit 79. In this case, the operation area 105 is displayed on the display unit 73. In the
operation area 105, an input operation unit 107 (text box or combo box) capable of inputting
numerical values with a keyboard or a mouse is arranged.
[0058]
03-05-2019
17
In the step of adjusting the horizontal angle in the sound collection direction of the microphone
array 15, the PC 21 calculates an angle between the camera reference line 95 and the
microphone reference line 97 as a deviation amount, and uses this deviation amount to calculate
Adjust the horizontal angle in the sound collection direction. Thereby, when forming directivity,
for example, the PC 21 corrects the reference direction of the horizontal angle in the sound
collection direction of the microphone array 15 using the amount of deviation, and the direction
indicated by the value after the correction (that is, all Since the same direction as the reference
direction of the horizontal angle of the imaging direction of the azimuth camera 13 can be used,
directivity can be appropriately formed in a state where the imaging direction of the
omnidirectional camera 13 and the reference match.
[0059]
FIG. 7 is a flow chart for explaining the processing procedure of the omnidirectional camera at
the time of calibration setting.
[0060]
As a processing procedure of the omnidirectional camera 13, first, a video signal is transmitted
from the communication unit 87 to the network 19 (st1).
Next, information on the reference direction of the horizontal angle of the imaging direction of
the omnidirectional camera 13 is transmitted to the PC 21 via the network 19 (st2). The video
signal and the information on the reference direction of the horizontal angle of the imaging
direction of the omnidirectional camera 13 may be simultaneously transmitted (see dotted line).
The omnidirectional camera 13 receives the information on the adjustment position sent from
the PC 21 (that is, the information on the reference direction of the horizontal angle of the sound
collection direction of the microphone array 15 after being adjusted by the end of the calibration
method of this embodiment) Wait and wait (st3). When the information on the adjustment
position in step st3 is received by the omnidirectional camera 13, the omnidirectional camera 13
determines the information on the adjustment position (st4), and stores the information on the
adjustment position in the storage unit 89.
[0061]
FIG. 8 is a flowchart for explaining the processing procedure of the PC 21 at the time of
03-05-2019
18
calibration setting.
[0062]
As the processing procedure of the PC 21, first, the PC 21 receives the video data, and then
displays the captured image 99 on the display unit 73 (st5).
Subsequently, the PC 21 receives the information on the reference direction of the horizontal
angle of the imaging direction of the omnidirectional camera 13 stored in the storage unit 89 of
the omnidirectional camera 13, and then the reference direction of the horizontal angle of the
imaging direction of the omnidirectional camera 13. The camera reference line 95 indicating the
image is superimposed and displayed on the captured image 99 (st6). The captured image 99
and the camera reference line 95 may be displayed simultaneously (see dotted line).
[0063]
The PC 21 waits for designation of the position of the marker 61 indicating the reference
direction of the horizontal angle in the sound collection direction of the microphone array 15
(st7). When the position of the marker 61 is designated, the PC 21 displays a broken line in the
radial direction from the image center 101 to the marker 61 (st 8). The microphone reference
line 97 indicating the reference direction of the horizontal angle in the sound collection direction
of the microphone array 15 is determined (st9), and when the setting button 103 is clicked by
the user, the microphone reference line 97 changes from a broken line to a solid line. The
reference direction of the horizontal angle of 15 sound collection directions is determined.
Information on the determined reference direction (adjustment position) of the horizontal angle
in the sound collection direction of the microphone array 15 is transmitted from the PC 21 to the
omnidirectional camera 13 and the microphone array 15 (st10).
[0064]
Next, the operation of the above-described image pickup and sound pickup system 11 will be
described.
[0065]
03-05-2019
19
In the calibration method of the present embodiment, in the omnidirectional camera 13, a lens
unit in which a lens unit and an imaging device are integrated is rotatably attached to a housing
51.
After the omnidirectional camera 13 and the microphone array 15 are attached to a horizontal
surface such as a ceiling surface in a state in which the reference directions of the horizontal
angles coincide with each other, a captured image of the omnidirectional camera 13 is displayed
on the display unit 73 of the PC 21 When the adjusting screw 59 is rotated by the tool and the
lens unit is rotated so that the user can obtain the desired angle of view in the state, the
reference directions of the horizontal angles of the omnidirectional camera 13 and the
microphone array 15 coincide with each other. It disappears.
[0066]
Therefore, in the calibration method of the present embodiment, when the captured image 99 of
the omnidirectional camera 13 is displayed on the display unit 73 of the PC 21, a camera
reference indicating the reference direction of the horizontal angle of the imaging direction of
the omnidirectional camera 13 Line 95 is displayed.
[0067]
In the captured image 99, a part (sub cover 57) of the casing 51 of the omnidirectional camera
13 opened by the user is displayed.
A marker 61 indicating a reference position (reference direction) of a horizontal angle in the
sound collection direction of the microphone array 15 is provided in advance on a part of the
housing 51 (sub cover 57).
[0068]
The user places the cursor on the marker 61 on the display unit 73 of the PC 21 and designates
the position (for example, depressing a mouse button). Thereby, the omnidirectional camera 13
displays the microphone reference line 97 indicating the reference position (reference direction)
of the horizontal angle in the sound collection direction of the microphone array 15 in a broken
03-05-2019
20
line on the captured image 99.
[0069]
Therefore, the camera reference line 95 and the microphone reference line 97 drawn in the
radial direction from the image center 101 are displayed in the captured image 99.
[0070]
An amount of shift ω (in other words, the user) in the reference direction of the rotational
direction (horizontal angle) about the central axis 47 of the omnidirectional camera 13 and the
microphone array 15 between the camera reference line 95 and the microphone reference line
97 In order to improve the visibility of the captured image, the shift amount in the reference
direction of the horizontal angle of the sound collection direction of the microphone array 15
based on the rotation of the lens unit by the rotation of the adjustment screw 59 is obtained.
For example, when forming directivity of audio data in a direction designated by the user, the PC
21 corrects the horizontal angle in the sound collection direction of the microphone array 15 in
consideration of the deviation amount ω.
[0071]
Further, in this calibration method, immediately after the user designates the position of the
marker 61, the microphone reference line 97 is displayed by a broken line in the captured image
99. Thereby, the user can easily visually recognize that the input position is unconfirmed.
[0072]
The broken microphone reference line 97 is displayed in place of a solid line when the user
designates a confirmation instruction to the PC 21. Thereby, the user can easily visually
recognize that the calibration process is completed.
[0073]
03-05-2019
21
Furthermore, in this calibration method, when an angle is numerically input to the operation area
105 displayed together with the captured image 99, the microphone reference line 97 is redrawn
based on the input numerical value. That is, the microphone reference line 97 can be manually
corrected. This makes it possible to more accurately match the microphone reference line 97
with the marker 61.
[0074]
Therefore, according to the calibration method of the present embodiment, when the
omnidirectional camera 13 and the microphone array 15 are integrally used, the imaging and
sound pickup system 11 can easily perform the operation of the omnidirectional camera 13
when the omnidirectional camera 13 and the microphone array 15 are integrally used. The
reference direction of the horizontal angle of the imaging direction and the reference direction of
the horizontal angle of the sound collection direction of the microphone array 15 can be
adjusted.
[0075]
Although various embodiments have been described above with reference to the drawings, it
goes without saying that the present invention is not limited to such examples.
It will be apparent to those skilled in the art that various changes and modifications can be made
within the scope of the appended claims, and of course these also fall within the technical scope
of the present invention. It is understood.
[0076]
For example, in the embodiment described above, when adjusting the reference direction of the
horizontal angle of the sound collection direction of the microphone array 15, the camera
reference line 95 indicating the reference direction of the horizontal angle of the imaging
direction of the omnidirectional camera 13; The example has been described in which the
microphone reference line 97 directed from the image center 101 to the marker 61 is displayed
together with the camera reference line 95 and the microphone reference line 97.
[0077]
03-05-2019
22
However, the present embodiment is not limited to the above example. For example, the PC 21
does not display the camera reference line 95 and the microphone reference line 97 on the
display unit 73 and the omnidirectional camera 13 is displayed when the sub cover 57 is opened.
The captured marker 61 is superimposed on the captured image 99 and displayed.
When the user clicks the marker 61, the PC 21 recognizes the reference direction of the
horizontal angle in the sound collection direction of the microphone array 15 as the direction
from the image center 101 to the marker 61, and further, the horizontal direction of the
omnidirectional camera 13 in the imaging direction. Since the information on the reference
direction of the angle is held in the storage unit 75, the value of the difference between the
reference direction of the horizontal angle of the sound collection direction of the microphone
array 15 and the reference direction of the horizontal angle of the imaging direction of the
omnidirectional camera 13 It can hold. That is, even if the camera reference line 95 and the
microphone reference line 97 are not displayed on the display unit 73, the reference direction of
the horizontal angle of the sound collection direction of the microphone array 15 is adjusted as
in the calibration method of the present embodiment described above. it can.
[0078]
Further, according to the present invention, when the omnidirectional camera and the
microphone array are integrally used, the reference direction of the imaging direction of the
omnidirectional camera and the reference direction of the sound collecting direction of the
microphone array are matched by easy operation. The present invention is useful as a calibration
method for adjusting the reference direction of the sound collection direction of the microphone
array, or as a sound collection imaging system used for the calibration method.
[0079]
11 imaging / sound collection system 13 omnidirectional camera 15 microphone array 51
housing 57 sub cover (partial) 61 marker 79 operation unit 95 camera reference line 97
microphone reference line 99 captured image 101 image center ω shift amount
03-05-2019
23
Документ
Категория
Без категории
Просмотров
0
Размер файла
36 Кб
Теги
jp2016119621
1/--страниц
Пожаловаться на содержимое документа