Patent Translate Powered by EPO and Google Notice This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate, complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or financial decisions, should not be based on machine-translation output. DESCRIPTION JP2014165901 Abstract: To provide a sound field sound collecting and reproducing technique capable of reproducing a sound field with higher accuracy than conventional. A conversion filter unit 3 applies a filter F (ω) defined by the following equation to a spherical harmonic spectrum signal P (ω) generated based on a signal collected by a microphone. Then, the post-filtering signal D ~ (ω) is generated. The discrete spherical harmonic inverse transform unit 4 transforms the filtered signal D to (ω) into a frequency domain signal by discrete spherical harmonic inverse transform. The frequency converter 5 converts the frequency domain signal into a time domain signal by inverse Fourier transform, and outputs the converted time domain signal to the speaker. [Selected figure] Figure 1 Sound field sound collecting and reproducing apparatus, method and program [0001] This invention is a technology of wave field synthesis (Ambisonics) that collects a sound signal with a microphone installed in a certain sound field and reproduces the sound field with a speaker using that sound signal. On the technology of [0002] Wavefront synthesis and ambisonics are techniques for virtually reproducing the sound field of a remote place using a plurality of microphones and speakers. 03-05-2019 1 As such technology, for example, the technology described in Non-Patent Document 1 is known. Since applications such as remote communication systems require real-time sound collection and reproduction, the sound pressure collected by a general microphone array is unique to a sound field reproduction signal for output by a general speaker array. Need to be convertible to [0003] Oyama, Furuya, Kazukazu Kazukazu, Haneda, Suzuki, “Wavefront Reconstruction Filter for Cylindrical Microphone-Speaker Array”, September 2012, Proceedings of the Fall Meeting of the Acoustical Society of Japan, pp. 605-608 [0004] In the technique described in Non-Patent Document 1, a filter for conversion has been derived assuming that a microphone array and a speaker array arranged in a cylindrical shape are used. Therefore, if this filter is applied to a microphone array and a speaker array arranged in a spherical shape, the sound field can not be reproduced with high accuracy, and the sound field in which sound comes from all directions around the listener is made highly accurate. There was a possibility that it could not be reproduced. [0005] The object of the present invention is to reproduce a sound field with higher accuracy than in the prior art when using a microphone array / speaker array arranged in a spherical shape, and to make sound come from all directions around the listener. A sound field sound collecting and reproducing apparatus, method and program capable of reproducing a field with high accuracy. [0006] In order to solve the above-mentioned problems, the sound field collection and reproduction apparatus according to one aspect of the present invention has an outer surface on a spherical surface of radius Rm by fixing at least three microphones to a spherical rigid baffle of radius Rb. , And at least three speakers are disposed inward on a sphere of radius Rs, Rm, Rs R Rb, j is an imaginary unit, ω is a frequency, c Is the speed of sound, k = ω / c, n and m are the orders of the spherical harmonic spectrum, jn (·) is the n-th sphere Bessel function, and hn <(1)> (·) is the n-th first Let the seed sphere Hankel function, jn '(.) Be the derivative of jn (.), Hn <(1)>' (.) Be the derivative of hn <(1)> (.), A (ω) be the predetermined Defined as a complex number of w and w 03-05-2019 2 nm as a weight determined based on n and m, for a spherical harmonic spectrum signal P.about.nm (.omega.) Generated based on a signal collected by a microphone, A conversion filter unit that applies the filter F to nm (ω) to generate a filtered signal D to nm (ω); [0007] [0008] A discrete spherical harmonic inverse transform unit that transforms the filtered signal D ~nm (ω) into a frequency domain signal by discrete spherical harmonic inverse transform, and a frequency domain signal transformed into a time domain signal by inverse Fourier transform And a frequency inverse converter for outputting a time domain signal to the speaker. [0009] The sound field collection and reproduction apparatus according to another aspect of the present invention is disposed outward on a spherical surface of radius Rm by fixing at least three microphones to a rigid baffle of a spherical rigid material of radius Rb. Assuming that at least three speakers are disposed inward on a spherical surface of radius Rs, Rm, Rs R Rb, j is an imaginary unit, ω is a frequency, c is a speed of sound, k = ω Let n, m be the order of the spherical harmonic spectrum, jn (.) be the nth-order spherical Bessel function, hn <(1)> (.) be the nth-order first-class sphere Hankel function, jn Let '(.) Be the derivative of jn (.), Hn <(1)>' (.) Be the derivative of hn <(1)> (.), A (ω) be the complex number, and wnm be n , m as a weight, a frequency conversion unit that converts a signal picked up by the microphone into a frequency domain signal by Fourier transform, and a discrete frequency domain harmonic frequency transform Applying a discrete spherical harmonic transform unit for transforming into spherical harmonic spectrum signal P ~nm (ω) and a filter F ~nm (ω) defined by the following equation for spherical harmonic spectrum signal P ~nm (ω) And a conversion filter unit that generates the post-filtering signal D ~nm (ω). [0010] [0011] The sound field collection and reproduction apparatus according to another aspect of the present invention is disposed outward on a spherical surface of radius Rm by fixing at least three microphones to a rigid baffle of a spherical rigid material of radius Rb. Assuming that at least three speakers are disposed inward on a spherical surface of radius Rs, Rm = Rb, RsRRb, j is an imaginary unit, ω is a frequency, c is a speed of sound, k Let n and m be the orders of the 03-05-2019 3 spherical harmonic spectrum, and hn <(1)> (·) be the n-th kind first-class sphere Hankel function and hn <(1)> '(·) be hn A spherical harmonic spectrum generated based on a signal picked up by a microphone, where A (ω) is a predetermined complex number and w nm is a weight determined based on n and m, with a derivative of <(1)> (·) A conversion filter unit that generates a filtered signal D to nm (ω) by applying a filter F to nm (ω) defined by the following equation to the signal P to nm (ω); [0012] [0013] A discrete spherical harmonic inverse transform unit that transforms the filtered signal D ~nm (ω) into a frequency domain signal by discrete spherical harmonic inverse transform, and a frequency domain signal transformed into a time domain signal by inverse Fourier transform And a frequency inverse converter for outputting a time domain signal to the speaker. [0014] The sound field collection and reproduction apparatus according to another aspect of the present invention is disposed outward on a spherical surface of radius Rm by fixing at least three microphones to a rigid baffle of a spherical rigid material of radius Rb. Assuming that at least three speakers are disposed inward on a spherical surface of radius Rs, Rm = Rb, RsRRb, j is an imaginary unit, ω is a frequency, c is a speed of sound, k Let n and m be the orders of the spherical harmonic spectrum, and hn <(1)> (·) be the n-th kind first-class sphere Hankel function and hn <(1)> '(·) be hn A derivative of <(1)> (·), A (ω) is a predetermined complex number, w nm is a weight determined based on n and m, and a signal picked up by a microphone is converted to a frequency domain signal by Fourier transform Discrete spherical harmonic conversion unit for converting a frequency domain signal into a spherical harmonic spectrum signal P ~nm (ω) by means of a frequency conversion unit and a discrete spherical harmonic conversion A conversion filter unit that generates a filtered signal D ~nm (ω) by applying a filter F ~nm (ω) defined by the following equation to the spherical harmonic spectrum signal P ~nm (ω); Prepare. [0015] [0016] Using a filter for a microphone array / speaker array arranged in a sphere, the sound pickup signal of the microphone array arranged in a sphere is converted into a drive signal of a speaker array arranged in a sphere to reproduce a sound field Since it is possible, it becomes possible to 03-05-2019 4 reproduce with high precision a sound field in which sound comes from all directions around the listener. [0017] FIG. 2 is a functional block diagram showing an example of a sound field sound collecting and reproducing apparatus. The figure for demonstrating the example of arrangement | positioning of a microphone and a speaker. The figure for demonstrating the example of arrangement | positioning of a microphone and a speaker. The flowchart which shows the example of the sound field sound collection reproduction method. [0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the symbols “~”, “^”, etc. used in the text should originally be written directly above the previous character, but due to the limitations of the text notation Do. In the formula, these symbols are described at their original positions. Moreover, the processing performed in each element unit of a vector or a matrix is applied to all elements of the vector or the matrix unless otherwise noted. 03-05-2019 5 [0019] <Arrangement of Microphone Array and Speaker Array> As shown in FIG. 1, in the sound field collection and reproduction apparatus and method, Nch, θ × Nch, φ pieces arranged on the spherical surface of radius Rm of the first space Using a microphone array composed of microphones, and a speaker array composed of Nch, θ × Nch, φ speakers arranged on the spherical surface of radius Rs of the second space different from the first space, The sound field of the first space formed by the sound generated by the sound source So of the first space is reproduced in the second space. In FIG. 1, the sound source So reproduced in the second space is expressed as a sound source So '. The first space and the second space are mutually different spaces. Nch, θ, Nch, and φ are integers of 2 or more. [0020] There may be at least three microphones and at least three speakers. The number of microphones disposed in the first space and the number of speakers disposed in the second space may be different. When the number of microphones is larger than the number of speakers disposed in the second space, the reproduction signal may be thinned. On the other hand, when the number of microphones is smaller than the number of speakers arranged in the second space, the reproduction signal may be interpolated by averaging the channels. As a method of performing interpolation, for example, linear interpolation or sinc interpolation can be applied. 03-05-2019 6 [0021] For example, as shown in FIG. 3, the microphone is disposed outward on the spherical surface SM of radius Rm by being fixed to the spherical rigid baffle SB of radius Rb. In this case, the microphone is disposed at a position Rm away from the center of the spherical baffle SB of radius Rb. In other words, as Rm> Rb, the microphone is placed at a position (Rm−Rb) away from the surface of the spherical baffle SB. For example, the microphone is disposed by supporting it by a thin rod-like member that protrudes vertically from the circumferential surface of the baffle SB. [0022] As described later, Rm may be equal to Rb. In this case, the microphones are arranged on the spherical surface of the spherical baffle. [0023] The microphones may be spaced at any distance. That is, assuming that θ is a zenith angle and φ is an azimuth angle, an interval between adjacent microphones in the zenith angle direction, θc, and an interval between adjacent microphones in the azimuthal direction, φc can take arbitrary values. . However, the sound field can be reproduced with high accuracy by arranging the microphones at equal intervals, that is, setting the respective θc and φc to the same value. As described above, by making θc and φc equal angles, so-called equal angle sampling becomes possible. [0024] Further, in order to perform so-called Gaussian sampling, the microphones may be arranged at a point where the azimuthal direction φ is equiangular and the zenith angle direction θ is Pn <m> (cos θ) = 0. Furthermore, in order to perform so-called uniform sampling, the microphones may be arranged to have as uniform density as possible on the spherical surface, such as the vertices of a regular polygon. 03-05-2019 7 [0025] Here, P <m> n (·) is a Legendre 陪 function and is defined as follows. P n (·) represents a Legendre polynomial. [0026] [0027] The speakers are arranged in the same way as the microphones, but the speakers may be arranged in the air of the second space in acoustically transparent state, or arranged in the second space in acoustically non-transparent state. It is also good. The acoustically transparent state is a state in which the same transfer characteristic as the transfer characteristic of the second space in which the speaker is not disposed is maintained. [0028] For example, the speaker is placed in the air of the second space in an acoustically transparent state by being suspended by a thread or fixed by a thin rod. Also, the speaker is acoustically transparent by being disposed on a spherical surface of radius Rs by being fixed to a spherical rigid baffle SB of radius Rb as shown in FIG. In the second space. [0029] Also, while the microphone is disposed outward on the spherical surface of radius Rm, the speaker is disposed inwardly on the spherical surface of radius Rs. [0030] The loudspeakers, like the microphones, need not be strictly spaced if they are approximately equally spaced. 03-05-2019 8 That is, the spacing θc between the speakers adjacent in the zenith angle direction and the φc adjacent in the azimuth angle direction do not have to be exactly the same value, and may be approximately the same value. When equal angle sampling is performed, the sound field can be reproduced with high accuracy by arranging the speakers at substantially equal intervals as in the microphone, that is, setting the values of θc and φc to the same value. [0031] Also, in order to perform so-called Gaussian sampling, Pn <m> (·) is a Legendre power function, microphones are used, azimuthal direction φ is equiangular, zenith angle direction θ is Pn <m> (cos θ) = 0 Similarly, in the case of the speaker arrangement, the azimuth angle direction φ is equiangular and the zenith angle direction θ is arranged at a point where Pn <m> (cos θ) = 0. [0032] Furthermore, when microphones are disposed as equidistant as possible on the spherical surface, such as the vertices of a regular polygon, in order to perform so-called uniform sampling, the speakers are similarly arranged on the spherical surface, such as the vertices of the regular polygon. Arrange as evenly as possible. [0033] It is assumed that RssRm, but this may not be satisfied. Although Rm and Rs can reproduce a wider area as their values are larger, more microphones and speakers are required. It is desirable to set Rm and Rs experimentally in consideration of the frequency of the signal to be collected. For example, Rs = 1.5 m and Rm = 0.2 m. [0034] Further, the radius Rs of the sphere in which the speaker is disposed is, for example, about 1.5 m. 03-05-2019 9 Although Rm ≦ Rs or Rs ≦ Rm may be satisfied, when Rm ≦ Rs, the accuracy of sound field reproduction is improved. [0035] The position of the microphone Mi-j in the first space is expressed as (Rm, θm, i, φm, j) [i = 1, 2,..., Nch, θ, j = 1, 2, ..., Nch, φ] Do. The position of the speaker Si-j in the second space is expressed as (Rs, θs, i, φs, j) [i = 1, 2,..., Nch, θ, j = 1, 2, ..., Nch, φ] Do. [0036] <Sound Field Sound Collection and Reproduction Device> As shown in FIG. 1, the sound field sound collection and reproduction device includes a frequency conversion unit 1, a discrete spherical harmonic conversion unit 2, a conversion filter unit 3, a discrete spherical harmonic inverse conversion unit 4, and a frequency inverse conversion. For example, the processing of each step illustrated in FIG. 4 is performed. [0037] The microphones M1-1, M2-1,..., MNch, θ-Nch, and φ arranged in the first space pick up the sound emitted by the sound source S of the first space and generate a time domain signal. Generate The generated signal is sent to the frequency converter 1. A signal of time t collected in the microphone Mi-j of (Rm, φm, i, zm, j) is denoted as pij (t). [0038] <Frequency Converter 1> The frequency converter 1 Fourier-transforms the signal pij (t) collected by the microphones M1-1, M2-1,..., MNch, θ-Nch, and φ into a frequency domain signal Pij (ω). (Step S1). The generated frequency domain signal P ij (ω) is sent to the discrete spherical harmonic converter 2. ω is a frequency. For example, frequency domain signal P ij (ω) is generated by short time discrete Fourier transform. Of course, the frequency domain signal P ij (ω) may be generated by another existing method. Alternatively, the frequency domain signal Pij 03-05-2019 10 (ω) may be generated using a method such as overlap ad. When the input signal is long or when the signal is continuously input as in real time processing, processing is performed every frame, for example, every 10 ms. The frequency domain signal P ij (ω) is defined, for example, as follows. J in the argument of the function exp is an imaginary unit. [0039] [0040] <Discrete Spherical Harmonic Conversion Unit 2> The discrete spherical harmonic conversion unit 2 converts the frequency domain signal Pij (ω) into a spherical harmonic spectrum signal P ~nm (ω) by discrete spherical harmonic conversion (step S2). The spherical harmonic spectrum signal P ~nm (ω) is calculated for each ω. The converted spherical harmonic spectrum signal P ~nm (ω) is sent to the conversion filter unit 3. [0041] When the equal angle sampling is performed, the discrete spherical harmonic conversion unit 2 specifically calculates P to nm (ω) defined by the equation (1). [0042] [0043] β i is a value determined according to the sampling method. Yn <m> is a spherical harmonic function defined by the following equation, and n and m are orders of a spherical harmonic spectrum. 0It is ≦ n ≦ N and −N ≦ m ≦ N, and n and m are integers. Let a be a complex number, and a <*> means a complex conjugate of a. 03-05-2019 11 [0044] [0045] Equation (1) is an example of discrete spherical harmonic conversion, and conversion to the spherical harmonic spectrum region may be performed by another method. Further, the spherical harmonic spectrum signal P to nm (ω) may be determined by numerical calculation. [0046] <Transformation Filter Unit 3> The conversion filter unit 3 applies the filter F ~nm (ω) defined by the equation (2) to the spherical harmonic spectrum signal P ~nm (ω) and applies the filter processed signal D ~ nm (ω) is generated (step S3). The filtered signal D ~nm (ω) is sent to the discrete spherical harmonic inverse transform unit 4. [0047] [0048] In equation (2), c = ω / c is the wave number, where c is the speed of sound. A (ω) is a predetermined complex number for adjusting the frequency characteristic. For example, A (ω) = 1 + 0 × j = 1. Further, w nm is a predetermined weight for attenuating the evanescent wave, which is determined as follows based on n and m, for example. In the following equations, nc and mc are predetermined values and are cutoff values of n and m, respectively. nc and mc are set to values that suppress evanescent waves, for example. α and β are predetermined values for determining the smoothness of the cutoff, for example, 0.05. Of course, another weighting function may be used as wnm. 03-05-2019 12 [0049] [0050] h <(1)> n (·) is an n-th-order first-class sphere Hankel function. h <(1)> n '(.) is a derivative of the n-th first-class sphere Hankel function h <(1)> n (.). jn (·) is an nth order spherical Bessel function. jn '(.) is a derivative of the nth-order spherical Bessel function jn (.). h <(1)> n (.), h <(1)> n '(.), jn (.), jn' (.) are defined as follows. [0051] [0052] Let Hn <(1)> (·) be an n-th kind Hankel function of the n-th order, and Jn (·) be a Bessel function of the n-th order. Hn <(1)> (·), Jn (·) are defined as follows. Γ (z) is a gamma function and Yn (z) is a Neumann function. [0053] [0054] In addition, when Rb = Rm, Formula (2) can be simplified as follows. [0055] [0056] 03-05-2019 13 The discrete spherical harmonic inverse transform unit 4 transforms the filtered signal D ~nm (ω) into a frequency domain signal Dij (ω) by discrete spherical harmonic inverse transform (step S 4). The converted frequency domain signal Dij (ω) is sent to the frequency inverse transform unit 5. When the equal angle sampling is performed, the discrete spherical harmonic inverse transform unit 4 specifically calculates the frequency domain signal Dij (ω) defined by the equation (3). [0057] [0058] Equation (3) is an example of discrete spherical harmonic inverse transformation, and transformation to the frequency domain may be performed by another method. Alternatively, the frequency domain signal Dij (ω) may be obtained by numerical calculation. [0059] <Frequency Inverse Transform Unit 5> The frequency inverse transform unit 5 converts the frequency domain signal Dij (ω) into a time domain signal P <d> ij (t) by inverse Fourier transform (step S5). The time domain signal P <d> ij (t) obtained for each frame by the inverse Fourier transform is appropriately shifted and linearly summed to be a continuous time domain signal. As the inverse Fourier transform, an existing method such as a short time discrete inverse Fourier transform may be used. The time domain signal P <d> ij (t) is sent to the speakers Si-j, S2-1,..., SNch, θ-Nch, φ. 03-05-2019 14 [0060] The speaker arrays S1-1, S2-1,..., SNch, θ-Nch, and φ reproduce sound based on the time domain signal P <d> ij (t). Specifically, the speaker Si-j reproduces a sound based on the time domain signal P <d> ij (t) as i = 1,..., Nch, θ, j = 1,. Thereby, the sound field of the first space can be reproduced in the second space. [0061] In this way, using a filter for microphone array and speaker array arranged in a sphere, the sound pickup signal of the microphone array arranged in a sphere is converted into a drive signal for a speaker array arranged in a sphere, and the sound field is Since it is possible to reproduce, it becomes possible to reproduce with high precision a sound field in which sound comes from all directions around the listener. [0062] <Theoretical Background> Hereinafter, the reason why the filter F to nm (ω) is expressed as Expression (2) is described. [0063] The secondary sound source signal is D (rs ', ω), and the synthetic sound field by the secondary sound source is Psyn (r', ω). Assuming that the transfer function from the secondary sound source at position rs 'to position r' is G (r'-rs ', ω), Psyn (r', ω) is a convolution of the spherical harmonic spectrum region as follows expressed. [0064] [0065] Here, n and m are the orders of the spherical harmonic spectrum. 03-05-2019 15 [0066] In the sound collection field, it is assumed that a spherical rigid baffle of radius Rb is installed at the center of the spherical microphone array of radius Rm. Here, Rb ≦ Rm. Assuming that the incident sound field is Pinc (r ′, ω) and the scattered sound field is Psct (r ′, ω), the sound collection field Prcv (r ′, ω) can be written as follows. [0067] [0068] Since the sound pressure gradient is zero on the rigid surface, it can be written as follows. [0069] [0070] Here, Pinc (·) and Psct (·) are represented as follows in the spherical harmonic spectrum region. [0071] [0072] Substituting the equations (8) and (7) into the equation (6) gives the following. [0073] [0074] Therefore, the relationship between the scattered sound field and the incident sound field can be 03-05-2019 16 derived as follows. [0075] [0076] From Formula (5), Formula (7), Formula (8), and Formula (9), it becomes as follows. [0077] [0078] Therefore, the incident sound field P ^ inc, n <m> (ω) can be expressed as follows using the sound collection field P ^ rcv, n <m> (ω). [0079] [0080] Since the desired sound field P ~ des (•) corresponds to the incident sound field Pinc (•), from equation (10), P ~ des (•) is the spherical harmonic spectrum P ~ rcv, on the receiving sphere. It can be expressed as follows using n <m> (Rm, ω). [0081] [0082] From Equation (4) and Equation (11), the conversion equation can be derived as follows. [0083] [0084] G (r'-rs', ω) was assumed to be monopole characteristics as follows. 03-05-2019 17 [0085] [0086] When the radius of the microphone array and that of the baffle are the same, Rb = Rm, and equation (12) can be simplified as follows. [0087] [0088] Here, the following relational expressions were used. [0089] [0090] In addition, since it does not change essentially as a characteristic of a filter, you may multiply A ((omega)) and wnm by Formula (12) and Formula (13). Then, these equations agree with the equations (2) and (2 '), respectively. [0091] [Modifications, Etc.] Each part constituting the sound field sound collecting and reproducing apparatus may be provided in either the sound collecting apparatus arranged in the first space or the reproduction apparatus arranged in the second space. In other words, the processing of each of the frequency conversion unit 1, the discrete spherical harmonic conversion unit 2, the conversion filter unit 3, the discrete spherical harmonic inverse conversion unit 4, the frequency inverse conversion unit 5 and the window function unit 6 is performed in the first space. It may be performed by the arranged sound collecting device or may be performed by the reproduction device arranged in the second space. 03-05-2019 18 The signal generated by the sound collection device is transmitted to the reproduction device. [0092] The positions of the first space and the second space are not limited to those shown in FIG. The first space and the second space may be adjacent to or separated from each other. Also, the orientation of the first space and the second space may be any. [0093] As long as the sound field sound collecting and reproducing apparatus includes the conversion filter unit 3, it does not have to include other units. For example, the sound field sound collecting and reproducing apparatus may be configured of the conversion filter unit 3, the discrete spherical harmonic inverse conversion unit 4, and the frequency inverse conversion unit 5. Further, the sound field sound collecting and reproducing apparatus may be configured of the frequency conversion unit 1, the discrete spherical harmonic conversion unit 2, and the conversion filter unit 3. [0094] The processing of the frequency conversion unit 1 and the processing of the discrete spherical harmonic conversion unit 2 may be performed simultaneously. 03-05-2019 19 Similarly, the processing of the discrete spherical harmonic inverse transform unit 4 and the processing of the frequency inverse transform unit 5 may be performed simultaneously. Alternatively, the discrete spherical harmonic transform unit 2 and the discrete spherical harmonic inverse transform unit 4 may be interchanged. [0095] The sound field sound collecting and reproducing apparatus can be realized by a computer. In this case, the processing content of each part of this apparatus is described by a program. And each part in this apparatus is implement | achieved on a computer by running this program by computer. [0096] The program describing the processing content can be recorded in a computer readable recording medium. Further, in this embodiment, these devices are configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized as hardware. [0097] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. [0098] Reference Signs List 1 frequency conversion unit 2 discrete spherical harmonic conversion unit 3 conversion filter unit 4 discrete spherical harmonic inverse conversion unit 5 frequency inverse conversion unit 03-05-2019 20 03-05-2019 21

1/--страниц