while this invention is illustrated and described in a preferred embodiment , the invention may be produced in many different configurations . there is depicted in the drawings , and will herein be described in detail , a preferred embodiment of the invention , with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and the associated functional specifications for its construction and is not intended to limit the invention to the embodiment illustrated . those skilled in the art will envision many other possible variations within the scope of the present invention . the present invention records sounds for comparison with a database of potentially similar sounds . while the preferred embodiment comprises heart sounds compared with a database of heart sounds for disease and defect identification , the present invention may be used in other applications such as bird songs or human voices . fig1 shows a computer hardware system 100 used in implementing the method and program code of a preferred embodiment . computer system 100 comprises a processor 110 , memory 120 , microphone 130 , input / output interface 140 , and database 150 . memory 120 stores sounds recorded by microphone 130 . in one embodiment , the recorded sounds are heart sounds from a patient . input / output interface 140 communicates with database 150 , which stores a plurality of stored sounds . in one embodiment , the stored sounds comprise previously recorded heart sounds , which have been categorized according to diagnosis . after the similar heart sounds are identified from the database , their associated disease labels are used to form a distribution of related diseases for physician decision support . fig2 shows the overall operation of the preferred embodiment as performed by computer hardware system 100 . beginning with step 210 , microphone 130 records a sound and stores it in memory 120 . processor 110 preprocesses the sound in step 220 by selecting a time period of interest , generally corresponding to one period of a periodic sound , such as a single heart beat . in step 230 , processor 110 constructs a line segment approximation of the single period of the sound , as shown in fig4 . in step 240 , processor 110 defines an audio envelope around the line segment approximation , as shown in fig5 . in step 250 , processor 110 isolates fiducial points from the audio envelope , as shown in fig6 . in step 260 , processor 110 matches the fiducial points from audio envelope of the recorded sound with fiducial points from the database sounds obtained through input / output interface 140 . a more detailed discussion of the pre - processing and modeling the shape variation of heart sounds performed by processor 110 will now be presented . while normal heart sounds show good repetition , the periodicity is surprisingly difficult to spot for abnormal heart sounds where the repetitions can be nested or irregular ( arrhythmias ). simple auto - correlation is often insufficient for this purpose . a robust periodicity detector treats periodicity detection as the problem of recovering a shift / translation that best aligns two self - similar curves . consider a periodic curve g ( t ) with period t . then by definition g ( t )= g ( t + kt ) for all multiples k = 0 , 1 , 2 , . . . . consider a candidate period τ . form a curve f ( t ) by shifting g ( t ) by τ , i . e . then define a function r ( τ ) that records the number of curve features that can be verified to satisfy the periodicity condition based on the current estimate of the period as r ⁡ ( τ ) =  { g ⁡ ( t i ) }  max ⁢ { n - τ , τ } ⁢ ⁢ such ⁢ ⁢ that ⁢ ⁢  g ⁡ ( t i ) - f ⁡ ( t j )  ≤ ɛ ( 2 ) where n is the total number of points on the curve , t i and t j are time values of matching fiducial points , and ε is an error tolerance . the above function ( 2 ) can be computed in linear time in comparison to the quadratic time for the autocorrelation function . the function r ( τ ) shows peaks at precisely those shifts which correspond to periodic repetitions of the curve . the most likely period is then taken as the smallest τ with the most integer multiples in the allowed range of heart beats ( 40 - 180 beats / minute ). there may be more than one candidate for period , particularly when the periodic repetitions are nested . the present invention &# 39 ; s algorithm finds such overlapping repetitions and tests the dynamic time warping algorithm with each such choice . various algorithms are available in literature for envelop extraction from signals including homomorphic filtering . while these algorithms are less sensitive to noise - related fluctuations , they frequently extract the low frequency component of the signal rather than render a faithful approximation of the perceptual envelope . in the present invention &# 39 ; s approach , the heart sound ( within a single heart beat ) is modeled through a perceptual envelop . to extract the audio envelope curve , processor 110 performs noise filtering using wavelet filters to remove the hissing noise that comes from digital stethoscopes . processor 110 then forms a line segment approximation to the audio signal . this is a standard split - and - merge algorithm for line segment approximation that uses two thresholds , namely , a distance threshold δ and a line length threshold l to recursively partition the audio signal into a set of line segments . each consecutive pair of line segments then defines a corner feature c i . with δ = 0 . 01 and l = 5 , a faithful rendering of the audio signal is made possible through a line segment approximation while retaining only 10 % of the samples . the thresholds for curve parameterization are not as critical here as the shape matching algorithm presented below , which is robust to missing and spurious features . the audio envelope ( ae ) is defined as the set of points ae ={ p i } where p i = c j for some j such that p i ( y )≧ p i − 1 ( y )& amp ; p i ( y )≧ p i + 1 ( y ) and p i ( y )≧ baseline . ( 1 ) only the values of the signal above the baseline are retained in the maxima envelope curve . similarly , all peaks below the baseline for the minima envelope curve . once the envelope curve f ( t ) is extracted , its shape can be represented by the curvature change points or corners on the envelope curve . the shape information at each corner is captured using the following parameters , wherein these parameters are chosen to facilitate matching of audio envelopes : s ( { right arrow over ( f )} ( t i ))=& lt ; t i , { right arrow over ( f )} ( t i ), θ ( t i ), φ ( t i )& gt ; ( 2 ) where θ ( t i ) is the included angle in the corner at t i , and φ ( t i ) is the orientation of the bisector at corner t i . using the angle of the corner ensures that wider complexes are not matched to narrow complex as these can change the disease interpretation . the angular bisector , on the other hand , ensures that polarity reversals such as inverted waves can be captured . referring to fig6 , consider an envelope curve g ( t ) corresponding to a heart sound . consider another curve f ( t ) that is a potential match to g ( t ), i . e . comes from a different patient diagnosed with the same disease . the curve f ( t ) is considered perceptually similar to g ( t ) if a non - rigid transform characterized by [ a , b , γ ] can be found such that where | | represents the distance metric that measures the difference between f ′( t ) and g ( t ), the simplest being the euclidean norm and f ( t )= af ( φ )( t )) with φ ( t )= bt + γ ( t ) ( 4 ) where the ( bt ) is the linear component of the transform and γ is the non - linear translation component . the parameters a and b are recovered by normalizing in amplitude and time . normalizing in amplitude is done by transforming f ( t ) and g ( t ) such that f ^ ⁡ ( t ) = ⁢ f ⁡ ( t ) - f min ⁡ ( t ) f max ⁡ ( t ) - f min ⁢ ⁢ f ⁡ ( t ) ⁢ ⁢ and g ^ ⁡ ( t ) = ⁢ g ⁡ ( t ) - g min ⁡ ( t ) g max ⁡ ( t ) - g min ⁢ ⁢ f ⁡ ( t ) ( 5 ) so that a = 1 . to eliminate solving for b , normalization of the time axis is done by dividing the heart rate . suppose the sampling rate of points on the curve f ( t ) is fs . let a periodicity detection algorithm signal the heart rate period to be t 1 . dividing by the heart period samples , all time instants lie in the range [ 0 , 1 ]. thus the time normalization can be easily achieved as : { right arrow over ( f )} ( t )= { circumflex over ( f )} ( t / t 1 ); { right arrow over ( g )} ( t )= ĝ ( t / t 2 ) ( 6 ) where t 1 and t 2 are the heart beat durations of f ( t ) and g ( t ) respectively . with this time normalization , b = 1 . such amplitude and time normalization automatically makes the shape modeling invariant to amplitude variations in audio recordings , as well as variations in heart rate across patients . since the non - uniform translation γ is a function of t , computational overhead is avoided by recovering it at important fiducial points such as the corners , and the overall shape approximation is recovered by interpolation . let there be k features extracted from { right arrow over ( f )}( t ) as f k ={( t 1 ,{ right arrow over ( f )} 1 ( t 1 )), ( t 2 ,{ right arrow over ( f )} 2 ( t 2 )), . . . ( t k ,{ right arrow over ( f )} k ( t k ))} at time { t 1 , t 2 , . . . , t k } respectively . let there be m fiducial points extracted from { right arrow over ( g )}( t ) as g m ={( t ′ 1 ,{ right arrow over ( g )} 1 ( t ′ 1 )), ( t ′ 2 ,{ right arrow over ( g )} 2 ( t ′ 2 )), . . . ( t ′ m ,{ right arrow over ( g )} m ( t ′ m ))} at time { t ′ 1 , t ′ 2 , . . . t ′ m }, respectively . if there is a set of n matching fiducial points c γ ={( t i , t ′ j )}, then the non - uniform translation transform γ can be defined as : γ ⁡ ( t ) = { t i if ⁢ ⁢ t = t j ′ ⁢ ⁢ and ⁢ ⁢ t i , t j ′ ∈ c γ t r + ( t s - t r t l ′ - t k ′ ) ⁢ ( t - t k ′ ) where ⁢ ⁢ ( t r , t k ′ ) , ( t s , t l ′ ) ∈ c γ ( 7 ) and t ′ k is the highest of t ′ j ≦ t and t ′ l is the lowest of t ′ j ≧ t that have a valid mapping in c γ . other interpolation methods besides linear ( e . g ., spline ) are also possible . using equations 6 and 7 , the shape approximation error between the two curves is then given by : | f ′( t )− g ( t )|=| { right arrow over ( f )} ( γ ( t ))− { right arrow over ( g )} ( t )| ( 8 ) for each g ( t ), γ is selected such that it minimizes the approximation error in ( 6 ) while maximizing the size of match c γ . finding the best matching audio based on shape can then be formulated as finding the g ( t ) such that g best = arg ⁢ min g ⁢  f -& gt ; ⁡ ( γ ⁡ ( t ) ) - g -& gt ; ⁡ ( t )  ( 9 ) while choosing the best γ for each respective candidate match g ( t ). if the feature set f k , g m extracted from the respective curves is considered as sequences , the problem of computing the best γ reduces to finding the best global subsequence alignment using the dynamic programming principle . the best global alignment maximizes the match of the curve fragments while allowing for possible gaps and insertions . gaps and insertions correspond to signal fragments from feature set f k that don &# 39 ; t find a match in set g m and vice versa . in fact , the alignment can be computed using a dynamic programming matrix h where the element h ( i , j ) is the cost of matching up to the ith and jth element in the respective sequences . as more features find a match , the cost increases as little as possible . the dynamic programming step becomes : h i , j = min ⁢ { h i - 1 , j - 1 + d ⁡ ( f -& gt ; ⁡ ( t i ) , g -& gt ; ⁡ ( t j ′ ) ) h i - 1 , j + d ⁡ ( f -& gt ; ⁡ ( t i ) , 0 ) h i , j - 1 + d ⁡ ( 0 , g -& gt ; ⁡ ( t j ′ ) ) ( 10 ) with initialization as h o , o = 0 and h o , j =∞ and h i , 0 =∞ for all 0 & lt ; i ≦ k , and 0 & lt ; j ≦ m . here d (.) is the cost of matching the individual features described next . also , the first term represents the cost of matching the feature point { right arrow over ( f )}( t i ) to feature point { right arrow over ( g )}( t ′ j ) which is low if the features are similar . the second term represents the choice where no match is assigned to feature { right arrow over ( f )}( t i ). after the transformation is recovered , the similarity between two envelop curves is given by the cost function d ({ right arrow over ( f )}( t i ),{ right arrow over ( g )}( t ′ j )): d ⁡ ( f -& gt ; ⁡ ( t i ) , g -& gt ; ⁡ ( t j ′ ) ) = { ( t i - t j ′ ) 2 + ( f -& gt ; ⁡ ( t i ) - g -& gt ; ⁡ ( t j ′ ) ) 2 + ⁢ ( θ ⁡ ( t i ) - θ ⁡ ( t j ′ ) ) 2 + ( φ ⁡ ( t i ) - φ ⁡ ( t j ′ ) ) 2 if ⁢ ⁢  t i - t j ′  ≤ and ⁢ ⁢ ( f -& gt ; ⁡ ( t i ) - g -& gt ; ⁡ ( t j ′ ) ) 2 ≤ λ 2 ⁢  θ ⁡ ( t i ) - θ ⁡ ( t j ′ )  ≤ λ 3 ⁢  φ ⁡ ( t i ) - φ ⁡ ( t j ′ )  ≤ λ 4 ∞ otherwise ( 11 ) the thresholds ( λ 1 , λ 2 , λ 3 , λ 4 ) are determined through a prior learning phase in which the expected variations per disease class is noted . the cost function d ({ right arrow over ( f )}( t i ), 0 ) can be computed by substituting t ′ j = 0 , { right arrow over ( g )}( t ′ j )= 0 and θ ( t + j )= 0 , φ ( t ′ j )= 0 in equation 11 . the cost function d ( 0 ,{ right arrow over ( g )}( t ′ j )) can be similarly computed . thus using the present invention &# 39 ; s approach , two heart sounds are considered similar if enough number of fiducial points between query and target envelop curves can be matched using shape - based dynamic time warping . in general , due to the period estimation offset errors , the signals may have to be circularly shifted by a fixed translation for a rough alignment before the fine non - rigid alignment described above . additionally , the present invention provides for an article of manufacture comprising computer readable program code contained within implementing one or more modules to detect audio similarity of sounds . furthermore , the present invention includes a computer program code - based product , which is a storage medium having program code stored therein which can be used to instruct a computer to perform any of the methods associated with the present invention . the computer storage medium includes any of , but is not limited to , the following : cd - rom , dvd , magnetic tape , optical disc , hard drive , floppy disk , ferroelectric memory , flash memory , ferromagnetic memory , optical storage , charge coupled devices , magnetic or optical cards , smart cards , eeprom , eprom , ram , rom , dram , sram , sdram , or any other appropriate static or dynamic memory or data storage devices . implemented in computer program code based products are software modules for recording a sound , selecting a time duration of a portion of the first sound to use as a second sound , constructing a line segment approximation of the second sound , defining an audio envelope around the second sound using the line segment approximation , isolating a plurality of fiducial points on the audio envelope , and matching the second sound to a plurality of similar sounds , wherein the matching comprises : identifying a non - rigid alignment transform based on a shape - based dynamic time warping to determine a correspondence between the fiducial points of the audio envelope and fiducial points from a database of sounds , defining a measure of shape similarity by determining a ratio of matched fiducial points to a total number of fiducial points , and ranking the matching based on the ratio . a system and method has been shown in the above embodiments for the effective implementation of a method and apparatus for retrieval of sounds from a database . while various preferred embodiments have been shown and described , it will be understood that there is no intent to limit the invention by such disclosure , but rather , it is intended to cover all modifications falling within the spirit and scope of the invention , as defined in the appended claims . for example , the present invention should not be limited by the type of audio . the above enhancements are implemented in various computing environments . for example , the present invention may be implemented on a conventional ibm pc or equivalent , multi - nodal system ( e . g ., lan ) or networking system ( e . g ., internet , www , wireless web ). all programming and data related thereto are stored in computer memory , static or dynamic , and may be retrieved by the user in any of : conventional computer storage , display ( i . e ., crt ) and / or hardcopy ( i . e ., printed ) formats . the programming of the present invention may be implemented by one of skill in the art of audio analysis .
the present invention exploits a visual rendering of heart sounds and models the morphological variations of audio envelopes through a constrained non - rigid translation transform . similar heart sounds are then retrieved by recovering the corresponding alignment transform using a variant of shape - based dynamic time warping .