init trans commit

.streamlit/config.toml

@@ -0,0 +1,2 @@
+[server]
+maxUploadSize = 10

Dockerfile

@@ -1,11 +1,24 @@
-FROM python:3.9
+FROM python:3.8-slim-buster
 
-WORKDIR /code
+# 將工作目錄設定為 /app
+WORKDIR /app
 
-COPY ./requirements.txt /code/requirements.txt
+# 複製當前目錄下的所有檔案到 /app
+COPY . /app
 
-RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
+# 開啟 8501 port
+EXPOSE 8501
 
-COPY . .
+# 安裝套件
+RUN apt update && \
+    apt upgrade -y && \
+    apt install -y libsndfile1 && \
+    apt install -y ffmpeg
 
-CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "7860"]
+# 升級 pip, 安裝套件
+RUN pip install --upgrade pip && \
+    pip install -r requirements.txt
+
+
+# 執行命令
+CMD ["streamlit", "run", "home.py"]

docs/1-Basic Information.md

@@ -0,0 +1,40 @@
+# Part1-Basic Information
+
+## What can it do?
+- Print audio length (seconds)
+- Plot waveform
+- Plot rms (librosa.feature.rms)
+- Plot spectrogram
+- Save rms as .csv
+
+## How to use it?
+
+### Step1：上傳音檔
+在上傳音檔區塊，點選上傳音檔，選擇要分析的音檔，點選上傳。
+> 上傳檔案限制：
+> - 檔案大小：200MB
+> - 檔案格式：`.mp3`, `.wav`, `.ocg`
+
+![](../fig/1-上傳檔案.png)
+
+上傳完成後，會顯示音檔的基本資訊，並提供一個播放介面檢查音檔是否正確。
+![](../fig/1-上傳完成.png)
+
+### Step2：選擇要分析的片段
+預設分析為整段音檔，若要分析音檔的特定片段，可以在此區塊選擇要分析的片段。
+同樣的，選擇完後，會顯示音檔的基本資訊，並提供一個播放介面檢查音檔是否正確。
+![](../fig/1-截取片段.png)
+
+### Step3：前往要分析的功能
+
+#### 聲音波形(waveform)
+![](../fig/1-繪製聲音波形.png)
+
+### 聲音強度(rms)
+![](../fig/1-聲音強度.png)
+
+### 聲音頻譜(spectrogram)
+![](../fig/1-聲音頻譜.png)
+
+### 儲存聲音強度(rms)
+![](../fig/1-下載RMS.png)

docs/2-Pitch_estimation.md

@@ -0,0 +1,15 @@
+# Part2-Pitch_estimation
+
+## What can it do?
+- Mel-frequency spectrogram
+- Constant-Q transform
+- Chroma
+- PYin + mel-frequency spectrogram
+- Pitch class histogram 1 (calculated from chroma_stft)
+- Pitch class histogram 2 (calculated from chroma_stft, n_chroma = 120)
+- Pitch class histogram 3 (calculated from the PYin f0 )
+- save all histogram (data count) as .csv
+
+## TODO
++ PYin + mel-frequency spectrogram 的部分是哪个？
++ 目前只有一个pirch class

docs/3-Beat Tracking.md

@@ -0,0 +1,18 @@
+# Part 3 - Beat Tracking
+
+## What can it do?
+-	Onset strength
+(librosa.onset.onset_strength)
+-	Onset detection + onset time
+(librosa.onset.onset_detect)
+-	Mel-spectrogram + onset strength + beat time
+-	Predominant local pulse + beat time 
+-	Tempo: print static tempo value
+-	Fourier tempogram + estimated tempo
+-	Autocorrelation tempogram + estimated tempo
+-	Save note onset time & beat time as .csv
+-	Output original audio + note onset click sound
+-	Output original audio + beat time click sound
+-	Madmom note onset detection & beat tracking
+https://madmom.readthedocs.io/en/latest/modules/features.html
+ 

docs/4-Chord_recognition.md

@@ -0,0 +1,6 @@
+# Part4-Chord recognition
+
+## What can it do?
+- Chromagram + chord recognition result
+(https://www.audiolabs-erlangen.de/resources/MIR/FMP/C5/C5S2_ChordRec_Templates.html)
+- Chromagram + binary reconstruction of chord recognition result

docs/5-Structure_analysis.md

@@ -0,0 +1,10 @@
+# Part5-Structure_analysis
+
+## What can it do?
+- Raw SSM (calculated from chroma)
+- Smoothed SSM (calculated from chroma)
+- Novelty function (structural boundary detection)
+- Save novelty function curve as .csv
+
+## TODO
++ 需要`anno.csv`的部分需要確認

docs/6-Timbre Analysis.md

@@ -0,0 +1,6 @@
+# Part 6 - Timbre analysis
+- Spectrogram + spectral centroid
+- Spectrogram + 99% roll-off + 1% roll-off
+- Spectrogram + centroid + bandwidth
+ (librosa.feature.spectral_bandwidth)
+- Save spectral centroid, 99% roll-off, 1% roll-off, bandwidth as .csv

docs/info.md

@@ -0,0 +1,55 @@
+# 音樂分析工具
+
+此工具整合Pitch_estimation、Beat Tracking、Chord recognition、Structure analysis和Timbre analysis等功能，旨在提供一個簡便易用的音樂分析工具。
+
+## 功能概述
+
+以下是此工具的主要功能：
+
+- Basic analysis：音檔基本資訊
+- Pitch estimation：樂曲音高估計
+- Beat Tracking：節奏追蹤
+- Chord recognition：和弦識別
+- Structure analysis：曲式分析
+- Timbre analysis：音色分析
+
+我們希望此專案可以幫助不具備程式基礎的音樂工作者和愛好者進行音樂分析，
+透過整合現有的各種音樂分析方法與工具，並將其整合在一個簡單易用的網頁工具介面中。
+
+## 開發團隊
+
++ [Yu-Fen Huang](https://yfhuang.info/)
+    + 中央研究院 資訊科學研究所 音樂與文化科技實驗室(Music & Culture Technology Lab, Institute of Information Science, Academia Sinica, Taiwan)
+    + 計畫主持人
++ [Yu-Lan Chang](https://github.com/TrangDuLam)
+    + 清華大學 電機工程所
+    + 核心功能開發、套件源碼整合
++ [Hong-Hsiang Liu](https://url.o-w-o.cc/link)
+    + 清華大學 電機工程所
+    + 互動介面設計、應用部署與配置
++ Ting-Yi Lu
+    + 清華大學 資訊工程所
+    + 套件源碼整合、說明文件撰寫
+
+## 工具相關資源
++ [視覺化介面](https://github.com/Keycatowo/music-analysis)：適合不具備程式基礎的使用者
++ [程式套件](https://github.com/TrangDuLam/NTHU_Music_AI_Tools)：提供更多細節調整，適合具備程式基礎的使用者
++ 說明文件：
+    + ...
+
+## 問題反饋
+
+如果您在使用此專案時遇到任何問題，請通過以下方式與我們聯繫：
+
+- 發送電子郵件給我們
+- 在我們的GitHub頁面提交問題
+    - [視覺化介面](https://github.com/Keycatowo/music-analysis/issues)
+    - [程式套件](https://github.com/TrangDuLam/NTHU_Music_AI_Tools)
+
+我們會盡快回复您的問題。
+
+## 授權協議
+
+音樂分析工具採用 [MIT](https://opensource.org/license/mit/) 授權。
+
+請注意，我們的軟件和內容可能包含第三方軟件庫和組件，這些庫和組件受到其各自的許可證的管轄。有關這些庫和組件的詳細信息，請參閱相應的文檔。

home.py

@@ -0,0 +1,67 @@
+#%%
+import streamlit as st
+
+st.header("Music Analysis Tool")
+
+st.session_state.start_time = 0.0
+
+
+st.write(
+    """
+    # 音樂分析工具
+
+此工具整合Pitch_estimation、Beat Tracking、Chord recognition、Structure analysis和Timbre analysis等功能，旨在提供一個簡便易用的音樂分析工具。
+
+## 功能概述
+
+以下是此工具的主要功能：
+
+- Basic analysis：音檔基本資訊
+- Pitch estimation：樂曲音高估計
+- Beat Tracking：節奏追蹤
+- Chord recognition：和弦識別
+- Structure analysis：曲式分析
+- Timbre analysis：音色分析
+
+我們希望此專案可以幫助不具備程式基礎的音樂工作者和愛好者進行音樂分析，
+透過整合現有的各種音樂分析方法與工具，並將其整合在一個簡單易用的網頁工具介面中。
+
+## 開發團隊
+
++ [Yu-Fen Huang](https://yfhuang.info/)
+    + 中央研究院 資訊科學研究所 音樂與文化科技實驗室(Music & Culture Technology Lab, Institute of Information Science, Academia Sinica, Taiwan)
+    + 計畫主持人
++ [Yu-Lan Chang](https://github.com/TrangDuLam)
+    + 清華大學 電機工程所
+    + 核心功能開發、套件源碼整合
++ [Hong-Hsiang Liu](https://url.o-w-o.cc/link)
+    + 清華大學 電機工程所
+    + 互動介面設計、應用部署與配置
++ Ting-Yi Lu
+    + 清華大學 資訊工程所
+    + 套件源碼整合、說明文件撰寫
+
+## 工具相關資源
++ [視覺化介面](https://github.com/Keycatowo/music-analysis)：適合不具備程式基礎的使用者
++ [程式套件](https://github.com/TrangDuLam/NTHU_Music_AI_Tools)：提供更多細節調整，適合具備程式基礎的使用者
++ 說明文件：
+    + ...
+
+## 問題反饋
+
+如果您在使用此專案時遇到任何問題，請通過以下方式與我們聯繫：
+
+- 發送電子郵件給我們
+- 在我們的GitHub頁面提交問題
+    - [視覺化介面](https://github.com/Keycatowo/music-analysis/issues)
+    - [程式套件](https://github.com/TrangDuLam/NTHU_Music_AI_Tools)
+
+我們會盡快回复您的問題。
+
+## 授權協議
+
+音樂分析工具採用 [MIT](https://opensource.org/license/mit/) 授權。
+
+請注意，我們的軟件和內容可能包含第三方軟件庫和組件，這些庫和組件受到其各自的許可證的管轄。有關這些庫和組件的詳細信息，請參閱相應的文檔。
+"""
+)

packages.txt

@@ -0,0 +1 @@
+libsndfile1-dev

pages/1-Basic_Information.py

@@ -0,0 +1,132 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+from src.st_helper import convert_df, show_readme, get_shift
+from src.basic_info import plot_waveform, signal_RMS_analysis
+
+
+st.title("Basic Information")
+#%% 頁面說明
+# show_readme("docs/1-Basic Information.md")
+
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+
+        y_all = y
+
+#%%
+if file is not None:
+
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+
+    tab1, tab2, tab3, tab4, tab5 = st.tabs([
+        "Waveform(mathplotlib)",
+        "Waveform(plotly)",
+        "signal_RMS_analysis",
+        "Spectrogram",
+        "Download RMS data"])
+    
+    shift_time, shift_array = get_shift(start_time, end_time) # shift_array為y_sub的時間刻度
+
+    # 繪製聲音波形圖
+    with tab1:
+        st.subheader("Waveform(mathplotlib)")
+        fig1_1, ax_1_1 = plt.subplots()
+        ax_1_1.plot(x_sub + shift_time, y_sub)
+        ax_1_1.set_xlabel("Time(s)")
+        ax_1_1.set_ylabel("Amplitude")
+        ax_1_1.set_title("Waveform")
+        st.pyplot(fig1_1)
+    
+    # 繪製聲音波形圖
+    with tab2:
+        st.subheader("Waveform(plotly)")
+        fig1_2 = go.Figure(data=go.Scatter(x=x_sub + shift_time, y=y_sub))
+        fig1_2.update_layout(
+            title="Waveform",
+            xaxis_title="Time(s)",
+            yaxis_title="Amplitude",
+        )
+        st.plotly_chart(fig1_2)
+
+    # 繪製聲音RMS圖
+    with tab3:
+        st.subheader("signal_RMS_analysis")
+        fig1_3, ax1_3, times, rms = signal_RMS_analysis(y_sub, shift_time=shift_time)
+        st.pyplot(fig1_3)   
+
+    # 繪製聲音Spectrogram圖(使用librosa繪製)
+    with tab4:
+        st.subheader("Spectrogram")
+        stft = librosa.stft(y_sub)
+        stft_db = librosa.amplitude_to_db(abs(stft))
+        # add a figure
+        fig1_4, ax1_4 = plt.subplots()
+        librosa.display.specshow(stft_db, x_axis='time', y_axis='log', sr=sr, ax=ax1_4)
+        ax1_4.set_xticks(shift_array - shift_array[0],
+                         shift_array)
+        ax1_4.autoscale()
+        ax1_4.set_xlabel("Time(s)")
+        st.pyplot(fig1_4)
+
+    # 下載RMS資料
+    with tab5:
+        st.subheader("Download RMS data")
+        
+        col1, col2 = st.columns(2)
+        with col1:
+            rms_df = pd.DataFrame({"Time(s)": times, "RMS": rms[0,:]})
+            st.dataframe(rms_df, use_container_width=True)
+        with col2:
+            st.download_button(
+                "Doanload RMS data",
+                convert_df(rms_df),
+                "rms.csv",
+                "text/csv",
+                key="download-csv"
+            )
+        
+# %%

pages/2-Pitch_estimation.py

@@ -0,0 +1,127 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+import seaborn as sns
+from src.st_helper import convert_df, show_readme, get_shift
+from src.pitch_estimation import plot_mel_spectrogram, plot_constant_q_transform, pitch_class_type_one_vis, pitch_class_histogram_chroma
+
+
+st.title("Pitch estimation")
+#%% 頁面說明
+# show_readme("docs/2-Pitch_estimation.md")
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+        
+        y_all = y
+
+#%% 功能區塊
+if file is not None:
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+
+    tab1, tab2, tab3, tab4 = st.tabs(["Mel-frequency spectrogram", "Constant-Q transform", "Chroma", "Pitch class"])
+
+    shift_time, shift_array = get_shift(start_time, end_time) # shift_array為y_sub的時間刻度
+
+    # Mel-frequency spectrogram
+    with tab1:
+        st.subheader("Mel-frequency spectrogram")
+        with_pitch = st.checkbox("Show pitch", value=True)
+        fig2_1, ax2_1 = plot_mel_spectrogram(y_sub, sr, shift_array, with_pitch)
+        st.pyplot(fig2_1)
+
+    # Constant-Q transform
+    with tab2:
+        st.subheader("Constant-Q transform")
+        fig2_2, ax2_2 = plot_constant_q_transform(y_sub, sr, shift_array)
+        st.pyplot(fig2_2)
+    
+    # chroma
+    with tab3:
+        st.subheader("Chroma")
+        
+        chroma = librosa.feature.chroma_stft(y=y_sub, sr=sr)
+        chroma_t = librosa.times_like(chroma, sr)
+        df_chroma = pd.DataFrame(chroma)
+        df_chroma_t = pd.DataFrame({"Time(s)": chroma_t})
+        df_chroma_t["Time(frame)"] = list(range(len(chroma_t)))
+        df_chroma_t["Time(s)"] = df_chroma_t["Time(s)"] + shift_time
+        df_chroma_t = df_chroma_t[["Time(frame)", "Time(s)"]]
+        
+        fig2_3, ax2_3 = plt.subplots(figsize=(10, 4))
+        sns.heatmap(chroma, ax=ax2_3)
+        ax2_3.set_title("Chroma")
+        ax2_3.set_xlabel("Time(frame)")
+        ax2_3.invert_yaxis()
+        st.pyplot(fig2_3)
+        
+        st.write("Chroma value")
+        st.dataframe(df_chroma, use_container_width=True)
+        st.download_button(
+            label="Download chroma",
+            data=convert_df(df_chroma),
+            file_name="chroma_value.csv",
+        )
+        st.write("Chroma time")
+        st.dataframe(df_chroma_t, use_container_width=True)
+        st.download_button(
+            label="Download chroma time",
+            data=convert_df(df_chroma_t),
+            file_name="chroma_time.csv",
+        )
+
+    # Pitch class type one
+    with tab4:
+        st.subheader("Pitch class(chroma)")
+        high_res = st.checkbox("High resolution", value=False)
+        fig2_4, ax2_4, df_pitch_class = pitch_class_histogram_chroma(y_sub, sr, high_res)
+        st.pyplot(fig2_4)
+        st.write(df_pitch_class)
+        st.download_button(
+            label="Download pitch class(chroma)",
+            data=convert_df(pd.DataFrame(df_pitch_class)),
+            file_name="Pitch_class(chroma).csv",
+            mime="text/csv",
+        )

pages/3-Beat Tracking.py

@@ -0,0 +1,164 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+from src.beat_track import onsets_detection, plot_onset_strength, beat_analysis, predominant_local_pulse, static_tempo_estimation, plot_tempogram, onset_click_plot, beat_plot
+from src.st_helper import convert_df, show_readme, get_shift
+import numpy as np
+
+st.title('Beat Tracking')
+
+#%% 頁面說明
+# show_readme("docs/3-Beat Tracking.md")
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+        
+        y_all = y
+
+#%% 功能區塊
+if file is not None:
+    
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+    
+    tab1, tab2, tab3, tab4, tab5, tab6 = st.tabs([
+        "onsets_detection",
+        "onset_strength",
+        "beat_analysis",
+        "predominant_local_pulse",
+        "static_tempo_estimation",
+        "Tempogram"])
+
+    shift_time, shift_array = get_shift(start_time, end_time) # shift_array為y_sub的時間刻度
+
+    # onsets_detection
+    with tab1:
+        st.subheader("onsets_detection")
+        fig3_1a, ax3_1a, onset_data = onsets_detection(y_sub, sr, shift_array)
+        o_env, o_times, onset_frames = onset_data
+        st.pyplot(fig3_1a)
+        # 設定onset_frame調整區塊
+        clicks = st.multiselect("Onset", 
+                                list(range(len(o_env))), list(onset_frames))
+        fig3_1b, ax3_1b, y_onset_clicks = onset_click_plot(o_env, o_times, clicks, len(y_sub), sr, shift_time)
+        st.pyplot(fig3_1b)
+        df_onset = pd.DataFrame({"Frame": clicks, "Time(s)": o_times[clicks], "Onset": o_env[clicks]})
+        st.dataframe(df_onset, use_container_width=True)
+        st.download_button(
+            label="Download onset data",
+            data=convert_df(df_onset),
+            file_name="onset_data.csv",
+        )
+        st.audio(y_onset_clicks, format="audio/ogg", sample_rate=sr)
+        
+
+    # onset_strength
+    with tab2:
+        st.subheader("onset_strength")
+        onset_strength_standard = st.checkbox("standard", value=True)
+        onset_strength_custom_mel = st.checkbox("custom_mel", value=False)
+        onset_strength_cqt = st.checkbox("cqt", value=False)
+        fig3_2, ax3_2 = plot_onset_strength(y_sub, sr,
+            standard=onset_strength_standard,
+            custom_mel=onset_strength_custom_mel,
+            cqt=onset_strength_cqt,
+            shift_array=shift_array
+        )
+        st.pyplot(fig3_2)
+
+    # beat_analysis
+    with tab3:
+        st.subheader("beat_analysis")
+        spec_type = st.selectbox("spec_type", ["mel", "stft"])
+        spec_hop_length = st.number_input("spec_hop_length", value=512)
+        fig3_3a, ax3_3b, beats_data = beat_analysis(y_sub, sr,
+            spec_type=spec_type,
+            spec_hop_length=spec_hop_length,
+            shift_array=shift_array
+        )
+        b_times, b_env, b_tempo, b_beats = beats_data
+        st.pyplot(fig3_3a)
+        
+        b_clicks = st.multiselect("Beats",
+                                  list(range(len(b_env))), list(b_beats))
+        fig3_3b, ax3_3b, y_beat_clicks = beat_plot(b_times, b_env, b_tempo, b_clicks, len(y_sub), sr, shift_time)
+        st.pyplot(fig3_3b)
+        # df_beats = pd.DataFrame([b_clicks, b_times[b_clicks] + shift_time])
+        # df_beats.index = ["frames", "time"]
+        df_beats = pd.DataFrame({"Frame": b_clicks, "Time(s)": b_times[b_clicks] + shift_time, "Beats": b_env[b_clicks]})
+        st.dataframe(df_beats, use_container_width=True)
+        st.download_button(
+            label="Download beats data",
+            data=convert_df(df_beats),
+            file_name="beats_data.csv",
+        )
+        st.audio(y_beat_clicks, format="audio/ogg", sample_rate=sr)
+
+
+    # predominant_local_pulse
+    with tab4:
+        st.subheader("predominant_local_pulse")
+        fig3_4, ax3_4 = predominant_local_pulse(y_sub, sr, shift_time)
+        st.pyplot(fig3_4)
+
+    # static_tempo_estimation
+    with tab5:
+        st.subheader("static_tempo_estimation")
+        static_tempo_estimation_hop_length = st.number_input("hop_length", value=512)
+        fig3_5, ax3_5 = static_tempo_estimation(y_sub, sr,
+            hop_length=static_tempo_estimation_hop_length
+        )
+        st.pyplot(fig3_5)
+
+    # Tempogram
+    with tab6:
+        st.subheader("Tempogram")
+        tempogram_type = st.selectbox("tempogram_type", ["fourier", "autocorr"], index=1)
+        tempogram_hop_length = st.number_input("Tempogram_hop_length", value=512)
+        fig3_6, ax3_6 = plot_tempogram(y_sub, sr,
+            type=tempogram_type,
+            hop_length=tempogram_hop_length,
+            shift_array=shift_array
+        )
+        st.pyplot(fig3_6)

pages/4-Chord_recognition.py

@@ -0,0 +1,112 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+import seaborn as sns
+from src.st_helper import convert_df, show_readme, get_shift
+from src.chord_recognition import (
+    plot_chord_recognition,
+    plot_binary_template_chord_recognition,
+    chord_table,
+    compute_chromagram,
+    chord_recognition_template,
+    plot_chord,
+    plot_user_chord
+)
+
+st.title("Chord Recognition")
+
+#%% 頁面說明
+# show_readme("docs/1-Basic Information.md")
+
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+
+        y_all = y
+
+#%%
+if file is not None:
+
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+
+    tab1, tab2, tab3, tab4 = st.tabs(["STFT Chroma", "Chords Result (Default)", "Chords Result (User)", "dev"])
+    shift_time, shift_array = get_shift(start_time, end_time) # shift_array為y_sub的時間刻度
+    
+    # STFT Chroma 
+    with tab1:
+        chroma, _, _, _, duration = compute_chromagram(y_sub, sr)
+        fig4_1, ax4_1 = plot_chord(chroma, "STFT Chroma")
+        st.pyplot(fig4_1)
+        
+    with tab2:
+        _, chord_max = chord_recognition_template(chroma, norm_sim='max')
+        fig4_2, ax4_2 = plot_chord(chord_max, "Chord Recognition Result", cmap="crest", include_minor=True)
+        st.pyplot(fig4_2)
+    
+    with tab3:
+        # 建立chord result dataframe
+        sec_per_frame = duration/chroma.shape[1]
+        chord_results_df = pd.DataFrame({
+            "Frame": np.arange(chroma.shape[1]),
+            "Time(s)": np.arange(chroma.shape[1])*sec_per_frame + shift_time,
+            "Chord": chord_table(chord_max)
+        })
+        
+        fig4_1b, ax4_1b = plot_user_chord(chord_results_df)
+        st.pyplot(fig4_1b)
+        
+        chord_results_df = st.experimental_data_editor(
+            chord_results_df,
+            use_container_width=True
+        )
+        
+
+    # plot_binary_template_chord_recognition
+    with tab4:
+        st.subheader("plot_binary_template_chord_recognition")
+        fig4_4, ax4_4 = plot_binary_template_chord_recognition(y_sub, sr)
+        st.pyplot(fig4_4)
+
+

pages/5-Structure_analysis.py

@@ -0,0 +1,76 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+from src.st_helper import convert_df, show_readme
+from src.structure_analysis import (
+    plot_self_similarity
+)
+
+st.title("Structure analysis")
+
+#%% 頁面說明
+# show_readme("docs/5-Structure_analysis.md")
+
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+        
+
+        y_all = y
+
+#%%
+if file is not None:
+
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+
+    tab1, tab2 = st.tabs(["Self-similarity matrix", "empty"])
+
+    # plot_self_similarity
+    with tab1:
+        st.subheader("Self-similarity matrix")
+        affinity = st.checkbox("Affinity", value=False)
+        self_similarity_hop_length = st.number_input("Self similarity hop length", value=1024)
+        fig5_1, ax5_1 = plot_self_similarity(y_sub, sr, affinity=affinity, hop_length=self_similarity_hop_length)
+        st.pyplot(fig5_1)

pages/6-Timbre_Analysis.py

@@ -0,0 +1,149 @@
+#%%
+import streamlit as st
+import plotly.express as px
+import plotly.graph_objects as go
+import matplotlib.pyplot as plt
+import numpy as np
+import librosa
+import pandas as pd
+from src.st_helper import convert_df, show_readme, get_shift
+from src.timbre_analysis import (
+    spectral_centroid_analysis,
+    rolloff_frequency_analysis,
+    spectral_bandwidth_analysis,
+    harmonic_percussive_source_separation
+)
+
+st.title("Timbre Analysis")
+#%% 頁面說明
+# show_readme("docs/6-Timbre Analysis.md")
+
+#%% 上傳檔案區塊
+with st.expander("上傳檔案(Upload Files)"):
+    file = st.file_uploader("Upload your music library", type=["mp3", "wav", "ogg"])
+
+    if file is not None:
+        st.audio(file, format="audio/ogg")
+        st.subheader("File information")
+        st.write(f"File name: `{file.name}`", )
+        st.write(f"File type: `{file.type}`")
+        st.write(f"File size: `{file.size}`")
+
+        # 載入音檔
+        y, sr = librosa.load(file, sr=44100)
+        st.write(f"Sample rate: `{sr}`")
+        duration = float(np.round(len(y)/sr-0.005, 2)) # 時間長度，取小數點後2位，向下取整避免超過音檔長度
+        st.write(f"Duration(s): `{duration}`")
+        
+        y_all = y
+
+#%%
+if file is not None:
+
+    ### Start of 選擇聲音片段 ###
+    with st.expander("選擇聲音片段(Select a segment of the audio)"):
+        
+        # 建立一個滑桿，可以選擇聲音片段，使用時間長度為單位
+        start_time, end_time = st.slider("Select a segment of the audio", 
+            0.0, duration, 
+            (st.session_state.start_time, duration), 
+            0.01
+        )
+        st.session_state.start_time = start_time
+
+    st.write(f"Selected segment: `{start_time}` ~ `{end_time}`, duration: `{end_time-start_time}`")
+
+    # 根據選擇的聲音片段，取出聲音資料
+    start_index = int(start_time*sr)
+    end_index = int(end_time*sr)
+    y_sub = y_all[start_index:end_index]
+
+        
+    # 建立一個y_sub的播放器
+    st.audio(y_sub, format="audio/ogg", sample_rate=sr)
+    # 計算y_sub所對應時間的x軸
+    x_sub = np.arange(len(y_sub))/sr
+    ### End of 選擇聲音片段 ###
+
+    tab1, tab2, tab3, tab4 = st.tabs(["Spectral Centroid", "Rolloff Frequency", "Spectral Bandwidth", "Harmonic Percussive Source Separation"])
+
+    shift_time, shift_array = get_shift(start_time, end_time) # shift_array為y_sub的時間刻度
+
+    # spectral_centroid_analysis
+    with tab1:
+        st.subheader("Spectral Centroid Analysis")
+        fig6_1, ax6_1, centroid_value = spectral_centroid_analysis(y_sub, sr, shift_array)
+        st.pyplot(fig6_1)
+        
+        df_centroid = pd.DataFrame(centroid_value.T, columns=["Time(s)", "Centroid"])
+        df_centroid["Time(s)"] = df_centroid["Time(s)"] + shift_time
+        st.dataframe(df_centroid, use_container_width=True)
+        st.download_button(
+            label="Download spectral centroid data",
+            data=convert_df(df_centroid),
+            file_name="centroid.csv",
+            mime="text/csv",
+        )
+
+    # rolloff_frequency_analysis
+    with tab2:
+        st.subheader("Rolloff Frequency Analysis")
+        roll_percent = st.selectbox("Select rolloff frequency", [0.90, 0.95, 0.99])
+        fig6_2, ax6_2, rolloff_value = rolloff_frequency_analysis(y_sub, sr, roll_percent=roll_percent, shift_array=shift_array)
+        st.pyplot(fig6_2)
+        df_rolloff = pd.DataFrame(rolloff_value.T, columns=["Time(s)", "Rolloff", "Rolloff_min"])
+        df_rolloff["Time(s)"] = df_rolloff["Time(s)"] + shift_time
+        st.dataframe(df_rolloff, use_container_width=True)
+        st.download_button(
+            label="Download rolloff frequency data",
+            data=convert_df(df_rolloff),
+            file_name="rolloff.csv",
+            mime="text/csv",
+        )
+
+    # spectral_bandwidth_analysis
+    with tab3:
+        st.subheader("Spectral Bandwidth Analysis")
+        fig6_3, ax6_3, bandwidth_value = spectral_bandwidth_analysis(y_sub, sr, shift_array)
+        st.pyplot(fig6_3)
+        df_bandwidth = pd.DataFrame(bandwidth_value.T, columns=["Time(s)", "Bandwidth"])
+        df_bandwidth["Time(s)"] = df_bandwidth["Time(s)"] + shift_time
+        st.dataframe(df_bandwidth, use_container_width=True)
+        st.download_button(
+            label="Download spectral bandwidth data",
+            data=convert_df(df_bandwidth),
+            file_name="bandwidth.csv",
+            mime="text/csv",
+        )
+
+    # harmonic_percussive_source_separation
+    with tab4:
+        st.subheader("Harmonic Percussive Source Separation")
+        fig6_4, ax6_4, (Harmonic_data) = harmonic_percussive_source_separation(y_sub, sr, shift_array)
+        D, H, P, t = Harmonic_data
+        st.pyplot(fig6_4)
+
+        st.download_button(
+            label="Download Full power spectrogram data",
+            data=convert_df(pd.DataFrame(D)),
+            file_name="Full_power_spectrogram.csv",
+            use_container_width=True,
+        )
+        st.download_button(
+            label="Download Harmonic power spectrogram data",
+            data=convert_df(pd.DataFrame(H)),
+            file_name="Harmonic_power_spectrogram.csv",
+            use_container_width=True,
+        )
+        st.download_button(
+            label="Download Percussive power spectrogram data",
+            data=convert_df(pd.DataFrame(P)),
+            file_name="Percussive_power_spectrogram.csv",
+            use_container_width=True,
+        )
+        st.download_button(
+            label="Download Time data",
+            data=convert_df(pd.DataFrame(t+shift_time, columns=["Time(s)"])),
+            file_name="Time_scale.csv",
+            use_container_width=True,
+        )

pages/999-dev.py

@@ -0,0 +1,18 @@
+#%%
+import pkg_resources
+import streamlit as st
+
+with st.expander("Show packages"):
+    for dist in pkg_resources.working_set:
+        print(f"{dist.project_name}=={dist.version}")
+        st.write(f"{dist.project_name}=={dist.version}")
+
+#%%
+import os
+import psutil
+
+with st.expander("Show memory usage"):
+    process = psutil.Process(os.getpid())
+    mem_info = process.memory_info()
+    print(f"Memory usage: {mem_info.rss / 1024 / 1024:.2f} MB")
+    st.write(f"Memory usage: {mem_info.rss / 1024 / 1024:.2f} MB")

requirements.txt

@@ -0,0 +1,11 @@
+librosa==0.9.2
+pandas==1.3.5
+streamlit==1.19.0
+numpy==1.23.0
+seaborn==0.12.1
+matplotlib==3.5.3
+plotly==5.11.0
+scikit-learn==1.2.0
+soundfile==0.11.0
+libfmp==1.2.3
+psutil==5.9.1

src/basic_info.py

@@ -0,0 +1,48 @@
+import librosa
+from librosa import display
+from librosa import feature
+
+import numpy as np
+from matplotlib import pyplot as plt
+import scipy
+
+from numpy import typing as npt
+import typing
+
+
+def show_duration(y: npt.ArrayLike, sr: int) -> float:
+    pass
+
+
+def selcet_time(start_time: float, end_time: float) :
+    pass
+
+
+def plot_waveform(ax, y: npt.ArrayLike, sr: int, start_time: float = 0.0, end_time: float = None) -> None :
+    # ax = plt.subplot(2, 1, 1)
+    startIdx = int(start_time * sr)
+    
+    if not end_time :
+        
+        librosa.display.waveshow(y[startIdx:], sr)
+    
+    else :
+        endIdx = int(end_time * sr)
+        librosa.display.waveshow(y[startIdx:endIdx - 1], sr)   
+    
+    return
+
+
+def signal_RMS_analysis(y: npt.ArrayLike, shift_time: float = 0.0) :
+
+    fig, ax = plt.subplots()
+
+    rms = librosa.feature.rms(y = y)
+    times = librosa.times_like(rms) + shift_time
+
+    ax.plot(times, rms[0])
+    ax.set_xlabel('Time (s)')
+    ax.set_ylabel('RMS')
+
+
+    return fig, ax, times, rms

src/beat_track.py

@@ -0,0 +1,223 @@
+import librosa
+from librosa import display
+from librosa import feature
+
+import numpy as np
+from matplotlib import pyplot as plt
+import scipy
+import soundfile as sf
+
+from numpy import typing as npt
+import typing
+
+
+def onsets_detection(y: npt.ArrayLike, sr: int, shift_array: npt.ArrayLike) -> tuple :
+    """
+        計算音檔的onset frames
+    """
+    o_env = librosa.onset.onset_strength(y=y, sr=sr)
+    times = librosa.times_like(o_env, sr=sr)
+    onset_frames = librosa.onset.onset_detect(onset_envelope=o_env, sr=sr)
+    D = np.abs(librosa.stft(y))
+
+    fig, ax = plt.subplots()
+    librosa.display.specshow(librosa.amplitude_to_db(D, ref=np.max),
+                             x_axis='time', y_axis='log', ax=ax, sr=sr)
+    ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+    ax.set_xlabel('Time (s)')
+    ax.autoscale()
+    ax.set(title='Power spectrogram')
+
+
+    return fig, ax, (o_env, times, onset_frames)
+
+def onset_click_plot(o_env, times, onset_frames, y_len, sr, shift_time) -> tuple:
+    """
+        重新繪製onset frames
+    """
+    fig, ax = plt.subplots()
+    ax.plot(times + shift_time, o_env, label='Onset strength')
+    ax.vlines(times[onset_frames] + shift_time, 0, o_env.max(), color='r', alpha=0.9,
+              linestyles='--', label='Onsets')
+    ax.autoscale()
+    ax.legend()
+    ax.set_xlabel('Time (s)')
+    ax.set_ylabel('Strength')
+    
+    y_onset_clicks = librosa.clicks(frames=onset_frames, sr=sr, length=y_len)
+    return fig, ax, y_onset_clicks
+    
+
+def plot_onset_strength(y: npt.ArrayLike, sr:int, standard: bool = True, custom_mel: bool = False, cqt: bool = False, shift_array: npt.ArrayLike = None) -> tuple:
+    
+    D = np.abs(librosa.stft(y))
+    times = librosa.times_like(D, sr)
+
+    fig, ax = plt.subplots(nrows=2, sharex=True)
+    librosa.display.specshow(librosa.amplitude_to_db(D, ref=np.max),
+                             y_axis='log', x_axis='time', ax=ax[0], sr=sr)
+    
+    ax[0].set(title='Power spectrogram')
+    ax[0].label_outer()
+
+    # Standard Onset Fuction 
+
+    if standard :
+        onset_env_standard = librosa.onset.onset_strength(y=y, sr=sr)
+        ax[1].plot(times, 2 + onset_env_standard / onset_env_standard.max(), alpha=0.8, label='Mean (mel)')
+    
+    if custom_mel :
+        onset_env_mel = librosa.onset.onset_strength(y=y, sr=sr,
+                                                     aggregate=np.median,
+                                                     fmax=8000, n_mels=256)
+        ax[1].plot(times, 1 + onset_env_mel / onset_env_mel.max(), alpha=0.8, label='Median (custom mel)')
+    
+    if cqt :
+        C = np.abs(librosa.cqt(y=y, sr=sr))
+        onset_env_cqt = librosa.onset.onset_strength(sr=sr, S=librosa.amplitude_to_db(C, ref=np.max))
+        ax[1].plot(times, onset_env_cqt / onset_env_cqt.max(), alpha=0.8, label='Mean (CQT)')
+
+    ax[1].legend()
+    ax[1].set(ylabel='Normalized strength', yticks=[])
+    ax[1].set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax[1].autoscale()
+    ax[1].set_xlabel('Time (s)')
+
+    return fig, ax
+
+
+def beat_analysis(y: npt.ArrayLike, sr:int, spec_type: str = 'mel', spec_hop_length: int = 512, shift_array: npt.ArrayLike = None) :
+    
+    fig, ax = plt.subplots()
+    onset_env = librosa.onset.onset_strength(y=y, sr=sr, aggregate=np.median)
+    tempo, beats = librosa.beat.beat_track(onset_envelope=onset_env, sr=sr)
+    times = librosa.times_like(onset_env, sr=sr, hop_length=spec_hop_length)
+
+    if spec_type == 'mel':
+        M = librosa.feature.melspectrogram(y=y, sr=sr, hop_length=spec_hop_length)
+        librosa.display.specshow(librosa.power_to_db(M, ref=np.max), 
+                                 y_axis='mel', x_axis='time', hop_length=spec_hop_length,
+                                 ax=ax, sr=sr)
+        ax.set(title='Mel spectrogram')
+
+    if spec_type == 'stft':
+        S = np.abs(librosa.stft(y))
+        img = librosa.display.specshow(librosa.amplitude_to_db(S, ref=np.max), 
+                                       y_axis='log', x_axis='time', ax=ax, sr=sr)
+        
+        ax.set_title('Power spectrogram')
+        # fig.colorbar(img, ax=ax[0], format="%+2.0f dB")
+    
+    ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+    ax.autoscale()
+    ax.set_xlabel('Time (s)')
+    
+    
+    return fig, ax, (times, onset_env, tempo, beats)
+
+def beat_plot(times, onset_env, tempo, beats, y_len, sr, shift_time):
+    """
+        重新繪製beat
+    """
+    
+    fig, ax = plt.subplots()
+    ax.plot(times + shift_time, librosa.util.normalize(onset_env), label='Onset strength')
+    ax.vlines(times[beats] + shift_time, 0, 1, alpha=0.5, color='r', linestyle='--', label='Beats')
+    tempoString = 'Tempo = %.2f'% (tempo)
+    ax.plot([], [], ' ', label = tempoString)
+    ax.legend()
+    ax.set_xlabel('Time (s)')
+    ax.set_ylabel('Normalized strength')
+    
+    y_beats = librosa.clicks(frames=beats, sr=sr, length=y_len)
+    
+    return fig, ax, y_beats
+
+def predominant_local_pulse(y: npt.ArrayLike, sr:int, shift_time:float=0) -> tuple :
+
+    onset_env = librosa.onset.onset_strength(y=y, sr=sr)
+    pulse = librosa.beat.plp(onset_envelope=onset_env, sr=sr)
+    beats_plp = np.flatnonzero(librosa.util.localmax(pulse))
+    times = librosa.times_like(pulse, sr=sr)
+
+    fig, ax = plt.subplots()
+    ax.plot(times + shift_time, librosa.util.normalize(pulse),label='PLP')
+    ax.vlines(times[beats_plp] + shift_time, 0, 1, alpha=0.5, color='r', 
+             linestyle='--', label='PLP Beats')
+    ax.legend()
+    ax.set(title="Predominant local pulse")
+    ax.set_xlabel('Time (s)')
+    ax.set_ylabel('Normalized strength')
+
+    return fig, ax
+
+
+def static_tempo_estimation(y: npt.ArrayLike, sr: int, hop_length: int = 512) -> tuple:
+  
+  '''
+  To visualize the result of static tempo estimation
+  
+  y: input signal array
+  sr: sampling rate
+  
+  '''
+
+  onset_env = librosa.onset.onset_strength(y=y, sr=sr)
+  tempo = librosa.beat.tempo(onset_envelope=onset_env, sr=sr)
+
+  # Static tempo estimation
+  prior = scipy.stats.uniform(30, 300)  # uniform over 30-300 BPM
+  utempo = librosa.beat.tempo(onset_envelope=onset_env, sr=sr, prior=prior)
+
+  tempo = tempo.item()
+  utempo = utempo.item()
+  ac = librosa.autocorrelate(onset_env, max_size=2 * sr // hop_length)
+  freqs = librosa.tempo_frequencies(len(ac), sr=sr,
+                                   hop_length=hop_length)
+
+  fig, ax = plt.subplots()
+  ax.semilogx(freqs[1:], librosa.util.normalize(ac)[1:],
+              label='Onset autocorrelation', base=2)
+  ax.axvline(tempo, 0, 1, alpha=0.75, linestyle='--', color='r',
+             label='Tempo (default prior): {:.2f} BPM'.format(tempo))    
+  ax.axvline(utempo, 0, 1, alpha=0.75, linestyle=':', color='g',
+             label='Tempo (uniform prior): {:.2f} BPM'.format(utempo)) 
+  ax.set(xlabel='Tempo (BPM)', title='Static tempo estimation')
+  ax.grid(True)
+  ax.legend() 
+
+  return fig, ax
+
+
+def plot_tempogram(y: npt.ArrayLike, sr: int, type: str = 'autocorr', hop_length: int = 512, shift_array: npt.ArrayLike = None) -> tuple :
+    
+    oenv = librosa.onset.onset_strength(y=y, sr=sr, hop_length=hop_length)
+    tempogram = librosa.feature.fourier_tempogram(onset_envelope=oenv, sr=sr, hop_length=hop_length)
+    tempo = librosa.beat.tempo(onset_envelope=oenv, sr=sr, hop_length=hop_length)[0]
+
+    fig, ax = plt.subplots()
+
+    if type == 'fourier' :
+        # To determine which temp to show?
+        librosa.display.specshow(np.abs(tempogram), sr=sr, hop_length=hop_length, 
+                                 x_axis='time', y_axis='fourier_tempo', cmap='magma')
+        ax.axhline(tempo, color='w', linestyle='--', alpha=1, label='Estimated tempo={:g}'.format(tempo))
+        ax.legend(loc='upper right')
+        # ax.title('Fourier Tempogram')
+
+    if type == 'autocorr' :
+        ac_tempogram = librosa.feature.tempogram(onset_envelope=oenv, sr=sr, hop_length=hop_length, norm=None)
+        librosa.display.specshow(ac_tempogram, sr=sr, hop_length=hop_length, x_axis='time', y_axis='tempo', cmap='magma')
+        ax.axhline(tempo, color='w', linestyle='--', alpha=1, label='Estimated tempo={:g}'.format(tempo))
+        ax.legend(loc='upper right')
+        # ax.title('Autocorrelation Tempogram')
+    ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+    ax.autoscale()
+    
+    return fig, ax
+
+

src/chord_recognition.py

@@ -0,0 +1,285 @@
+import os
+import numpy as np
+from matplotlib import pyplot as plt
+
+import librosa
+import libfmp.b
+import libfmp.c3
+import libfmp.c4
+
+import sys
+
+def compute_chromagram_from_filename(fn_wav, Fs=22050, N=4096, H=2048, gamma=None, version='STFT', norm='2'):
+    """Compute chromagram for WAV file specified by filename
+
+    Notebook: C5/C5S2_ChordRec_Templates.ipynb
+
+    Args:
+        fn_wav (str): Filenname of WAV
+        Fs (scalar): Sampling rate (Default value = 22050)
+        N (int): Window size (Default value = 4096)
+        H (int): Hop size (Default value = 2048)
+        gamma (float): Constant for logarithmic compression (Default value = None)
+        version (str): Technique used for front-end decomposition ('STFT', 'IIS', 'CQT') (Default value = 'STFT')
+        norm (str): If not 'None', chroma vectors are normalized by norm as specified ('1', '2', 'max')
+            (Default value = '2')
+
+    Returns:
+        X (np.ndarray): Chromagram
+        Fs_X (scalar): Feature reate of chromagram
+        x (np.ndarray): Audio signal
+        Fs (scalar): Sampling rate of audio signal
+        x_dur (float): Duration (seconds) of audio signal
+    """
+    x, Fs = librosa.load(fn_wav, sr=Fs)
+    x_dur = x.shape[0] / Fs
+    if version == 'STFT':
+        # Compute chroma features with STFT
+        X = librosa.stft(x, n_fft=N, hop_length=H, pad_mode='constant', center=True)
+        if gamma is not None:
+            X = np.log(1 + gamma * np.abs(X) ** 2)
+        else:
+            X = np.abs(X) ** 2
+        X = librosa.feature.chroma_stft(S=X, sr=Fs, tuning=0, norm=None, hop_length=H, n_fft=N)
+    if version == 'CQT':
+        # Compute chroma features with CQT decomposition
+        X = librosa.feature.chroma_cqt(y=x, sr=Fs, hop_length=H, norm=None)
+    if version == 'IIR':
+        # Compute chroma features with filter bank (using IIR elliptic filter)
+        X = librosa.iirt(y=x, sr=Fs, win_length=N, hop_length=H, center=True, tuning=0.0)
+        if gamma is not None:
+            X = np.log(1.0 + gamma * X)
+        X = librosa.feature.chroma_cqt(C=X, bins_per_octave=12, n_octaves=7,
+                                       fmin=librosa.midi_to_hz(24), norm=None)
+    if norm is not None:
+        X = libfmp.c3.normalize_feature_sequence(X, norm=norm)
+    Fs_X = Fs / H
+    return X, Fs_X, x, Fs, x_dur
+
+def compute_chromagram(y, sr, Fs=22050, N=4096, H=2048, gamma=None, version='STFT', norm='2'):
+    """Compute chromagram for WAV file specified by filename
+
+    Notebook: C5/C5S2_ChordRec_Templates.ipynb
+
+    Args:
+        y (np.ndarray): Audio signal
+        sr (scalar): Sampling rate
+        Fs (scalar): Sampling rate (Default value = 22050)
+        N (int): Window size (Default value = 4096)
+        H (int): Hop size (Default value = 2048)
+        gamma (float): Constant for logarithmic compression (Default value = None)
+        version (str): Technique used for front-end decomposition ('STFT', 'IIS', 'CQT') (Default value = 'STFT')
+        norm (str): If not 'None', chroma vectors are normalized by norm as specified ('1', '2', 'max')
+            (Default value = '2')
+
+    Returns:
+        X (np.ndarray): Chromagram
+        Fs_X (scalar): Feature reate of chromagram
+        x (np.ndarray): Audio signal
+        Fs (scalar): Sampling rate of audio signal
+        x_dur (float): Duration (seconds) of audio signal
+    """
+    x = librosa.resample(y, sr, Fs)
+    x_dur = x.shape[0] / Fs
+    if version == 'STFT':
+        # Compute chroma features with STFT
+        X = librosa.stft(x, n_fft=N, hop_length=H, pad_mode='constant', center=True)
+        if gamma is not None:
+            X = np.log(1 + gamma * np.abs(X) ** 2)
+        else:
+            X = np.abs(X) ** 2
+        X = librosa.feature.chroma_stft(S=X, sr=Fs, tuning=0, norm=None, hop_length=H, n_fft=N)
+    if version == 'CQT':
+        # Compute chroma features with CQT decomposition
+        X = librosa.feature.chroma_cqt(y=x, sr=Fs, hop_length=H, norm=None)
+    if version == 'IIR':
+        # Compute chroma features with filter bank (using IIR elliptic filter)
+        X = librosa.iirt(y=x, sr=Fs, win_length=N, hop_length=H, center=True, tuning=0.0)
+        if gamma is not None:
+            X = np.log(1.0 + gamma * X)
+        X = librosa.feature.chroma_cqt(C=X, bins_per_octave=12, n_octaves=7,
+                                       fmin=librosa.midi_to_hz(24), norm=None)
+    if norm is not None:
+        X = libfmp.c3.normalize_feature_sequence(X, norm=norm)
+    Fs_X = Fs / H
+    return X, Fs_X, x, Fs, x_dur
+
+def get_chord_labels(ext_minor='m', nonchord=False):
+    """Generate chord labels for major and minor triads (and possibly nonchord label)
+
+    Notebook: C5/C5S2_ChordRec_Templates.ipynb
+
+    Args:
+        ext_minor (str): Extension for minor chords (Default value = 'm')
+        nonchord (bool): If "True" then add nonchord label (Default value = False)
+
+    Returns:
+        chord_labels (list): List of chord labels
+    """
+    chroma_labels = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
+    chord_labels_maj = chroma_labels
+    chord_labels_min = [s + ext_minor for s in chroma_labels]
+    chord_labels = chord_labels_maj + chord_labels_min
+    if nonchord is True:
+        chord_labels = chord_labels + ['N']
+    return chord_labels
+
+def generate_chord_templates(nonchord=False):
+    """Generate chord templates of major and minor triads (and possibly nonchord)
+
+    Notebook: C5/C5S2_ChordRec_Templates.ipynb
+
+    Args:
+        nonchord (bool): If "True" then add nonchord template (Default value = False)
+
+    Returns:
+        chord_templates (np.ndarray): Matrix containing chord_templates as columns
+    """
+    template_cmaj = np.array([1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0]).T
+    template_cmin = np.array([1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0]).T
+    num_chord = 24
+    if nonchord:
+        num_chord = 25
+    chord_templates = np.ones((12, num_chord))
+    for shift in range(12):
+        chord_templates[:, shift] = np.roll(template_cmaj, shift)
+        chord_templates[:, shift+12] = np.roll(template_cmin, shift)
+    return chord_templates
+
+def chord_recognition_template(X, norm_sim='1', nonchord=False):
+    """Conducts template-based chord recognition
+    with major and minor triads (and possibly nonchord)
+
+    Notebook: C5/C5S2_ChordRec_Templates.ipynb
+
+    Args:
+        X (np.ndarray): Chromagram
+        norm_sim (str): Specifies norm used for normalizing chord similarity matrix (Default value = '1')
+        nonchord (bool): If "True" then add nonchord template (Default value = False)
+
+    Returns:
+        chord_sim (np.ndarray): Chord similarity matrix
+        chord_max (np.ndarray): Binarized chord similarity matrix only containing maximizing chord
+    """
+    chord_templates = generate_chord_templates(nonchord=nonchord)
+    X_norm = libfmp.c3.normalize_feature_sequence(X, norm='2')
+    chord_templates_norm = libfmp.c3.normalize_feature_sequence(chord_templates, norm='2')
+    chord_sim = np.matmul(chord_templates_norm.T, X_norm)
+    if norm_sim is not None:
+        chord_sim = libfmp.c3.normalize_feature_sequence(chord_sim, norm=norm_sim)
+    # chord_max = (chord_sim == chord_sim.max(axis=0)).astype(int)
+    chord_max_index = np.argmax(chord_sim, axis=0)
+    chord_max = np.zeros(chord_sim.shape).astype(np.int32)
+    for n in range(chord_sim.shape[1]):
+        chord_max[chord_max_index[n], n] = 1
+
+    return chord_sim, chord_max
+
+def plot_chord_recognition(y, sr) :
+    import warnings
+    warnings.warn("This function is deprecated and will be removed in future versions.", DeprecationWarning)
+    
+    X, Fs_X, x, Fs, x_dur = compute_chromagram(y, sr)
+    
+    chord_sim, chord_max = chord_recognition_template(X, norm_sim='max')
+    chord_labels = get_chord_labels(nonchord=False)
+
+    cmap = libfmp.b.compressed_gray_cmap(alpha=1, reverse=False)
+    fig, ax = plt.subplots(2, 2, gridspec_kw={'width_ratios': [1, 0.03], 
+                                          'height_ratios': [1.5, 3]}, figsize=(8, 10))
+
+    libfmp.b.plot_chromagram(X, ax=[ax[0,0], ax[0,1]], Fs=Fs_X, clim=[0, 1], xlabel='',
+                         title='STFT-based chromagram (feature rate = %0.1f Hz)' % (Fs_X))
+    libfmp.b.plot_matrix(chord_max, ax=[ax[1, 0], ax[1, 1]], Fs=Fs_X, 
+                     title='Time–chord representation of chord recognition result',
+                     ylabel='Chord', xlabel='')
+    ax[1, 0].set_yticks(np.arange( len(chord_labels) ))
+    ax[1, 0].set_yticklabels(chord_labels)
+    ax[1, 0].grid()
+    plt.tight_layout()
+    return fig, ax, chord_max
+
+def plot_binary_template_chord_recognition(y, sr) :
+    import warnings
+    warnings.warn("This function is deprecated and will be removed in future versions.", DeprecationWarning)
+    
+    X, Fs_X, x, Fs, x_dur = compute_chromagram(y, sr)
+    chord_sim, chord_max = chord_recognition_template(X, norm_sim='max')
+
+    chord_templates = generate_chord_templates()
+    X_chord = np.matmul(chord_templates, chord_max)
+
+    fig, ax = plt.subplots(2, 2, gridspec_kw={'width_ratios': [1, 0.03], 
+                                              'height_ratios': [1, 1]}, figsize=(8, 5))
+
+    libfmp.b.plot_chromagram(X, ax=[ax[0, 0], ax[0, 1]], Fs=Fs_X, clim=[0, 1], xlabel='',
+                            title='STFT-based chromagram (feature rate = %0.1f Hz)' % (Fs_X))
+    libfmp.b.plot_chromagram(X_chord, ax=[ax[1, 0], ax[1, 1]], Fs=Fs_X, clim=[0, 1], xlabel='',
+                            title='Binary templates of the chord recognition result')
+    plt.tight_layout()
+    return fig, ax
+
+
+def chord_table(chord_max):
+    
+    chord_labels = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'] + ['Cm', 'C#m', 'Dm', 'D#m', 'Em', 'Fm', 'F#m', 'Gm', 'G#m', 'Am', 'A#m', 'Bm']
+    
+    # 計算chord_max依照第一個軸的最大值的index
+    chord_max_index = np.argmax(chord_max, axis=0)
+    # 用index找出對應的chord_labels
+    chord_results = [chord_labels[i] for i in chord_max_index]
+    
+    return chord_results
+
+
+def plot_chord(chroma, title="", figsize=(12, 6), cmap="coolwarm", include_minor=False):
+    import seaborn as sns
+    chroma_labels = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
+    if include_minor:
+        chroma_labels += ['Cm', 'C#m', 'Dm', 'D#m', 'Em', 'Fm', 'F#m', 'Gm', 'G#m', 'Am', 'A#m', 'Bm']
+    
+    fig, ax = plt.subplots(figsize=figsize)
+    
+    sns.heatmap(chroma, ax=ax, cmap=cmap, linewidths=0.01, linecolor=(1, 1, 1, 0.1))
+    ax.invert_yaxis()
+    ax.set_yticks(
+        np.arange(len(chroma_labels)) + 0.5,
+        chroma_labels,
+        rotation=0,
+    )
+    ax.set_ylabel("Chord")
+    ax.set_xlabel('Time (frame)')
+    ax.set_title(title)
+    
+    return fig, ax
+
+def plot_user_chord(df):
+    
+    import seaborn as sns
+    chroma_labels = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'] + ['Cm', 'C#m', 'Dm', 'D#m', 'Em', 'Fm', 'F#m', 'Gm', 'G#m', 'Am', 'A#m', 'Bm']
+    
+    # 檢查df["Chord"]無chroma_labels以外的值
+    assert df["Chord"].isin(chroma_labels).all(), "Chord must be in chroma_labels"
+    
+    # 將df["Chord"]轉成chroma_labels的index
+    df["Chord_index"] = df["Chord"].apply(lambda x: chroma_labels.index(x))
+    
+    # 建立一個24 * len(df)的矩陣，並將值設為0
+    chroma = np.zeros((24, len(df)))
+    # 依照df["Chord_index"]的值將chroma的值設為1
+    chroma[df["Chord_index"], np.arange(len(df)),] = 1
+    
+    # 繪圖
+    fig, ax = plt.subplots(figsize=(12, 6))
+    sns.heatmap(chroma, ax=ax, cmap='crest', linewidths=0.01, linecolor=(1, 1, 1, 0.1))
+    ax.invert_yaxis()
+    ax.set_yticks(
+        np.arange(len(chroma_labels)) + 0.5,
+        chroma_labels,
+        rotation=0,
+    )
+    ax.set_ylabel("Chord")
+    ax.set_xlabel('Time (frame)')
+    ax.set_title('User Chord Recognition Result')
+    
+    return fig, ax

src/pitch_estimation.py

@@ -0,0 +1,181 @@
+import librosa
+from librosa import display
+from librosa import feature
+
+import numpy as np
+from matplotlib import pyplot as plt
+import scipy
+
+from numpy import typing as npt
+import typing
+
+
+def plot_mel_spectrogram(
+        y: npt.ArrayLike, 
+        sr:int, 
+        shift_array: npt.ArrayLike,
+        with_pitch : bool = True,
+    ):
+
+    S = librosa.feature.melspectrogram(y=y, sr=sr)
+    S_dB = librosa.power_to_db(S, ref=np.max)
+
+    if with_pitch :
+        
+        f0, voiced_flag, voiced_probs = librosa.pyin(y,
+                                                     fmin=librosa.note_to_hz('C2'),
+                                                     fmax=librosa.note_to_hz('C7'))
+        times = librosa.times_like(f0, sr)
+        
+        fig, ax = plt.subplots(figsize=(12,6))
+        img = librosa.display.specshow(S_dB, x_axis='time',
+                                       y_axis='mel', sr=sr, 
+                                       fmax=8000, ax=ax)
+        ax.plot(times, f0, label='f0', color='cyan', linewidth=3)
+        ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+        fig.colorbar(img, ax=ax, format='%+2.0f dB')
+        ax.legend(loc='upper right')
+        ax.set(title='Mel-frequency spectrogram')
+
+
+    else :
+        fig, ax = plt.subplots(figsize=(12,6))
+        img = librosa.display.specshow(S_dB, x_axis='time',
+                                       y_axis='mel', sr=sr, 
+                                       fmax=8000, ax=ax)
+        ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+        fig.colorbar(img, ax=ax, format='%+2.0f dB')
+        ax.set(title='Mel-frequency spectrogram')
+    ax.set_xlabel('Time (s)')
+    
+    return fig, ax
+
+
+def plot_constant_q_transform(y: npt.ArrayLike, sr:int,
+                              shift_array: npt.ArrayLike
+    ) :
+
+    C = np.abs(librosa.cqt(y, sr=sr))
+    fig, ax = plt.subplots(figsize=(12,6))
+    img = librosa.display.specshow(librosa.amplitude_to_db(C, ref=np.max),
+                                   sr=sr, x_axis='time', y_axis='cqt_note', ax=ax)
+    ax.set_xticks(shift_array - shift_array[0],
+                      shift_array)
+    ax.set_title('Constant-Q power spectrum')
+    ax.set_xlabel('Time (s)')
+    fig.colorbar(img, ax=ax, format="%+2.0f dB")
+
+    return fig, ax
+
+
+def pitch_class_type_one_vis(y: npt.ArrayLike, sr: int) -> None :
+
+    S = np.abs(librosa.stft(y))
+    chroma = librosa.feature.chroma_stft(S=S, sr=sr)
+
+    count_pitch = np.empty(np.shape(chroma)) # To count pitch
+    notes = np.array(librosa.key_to_notes('C:maj'))
+    
+    # Set the threshold to determine the exact pitch
+    count_pitch[chroma < 0.5] = 0
+    count_pitch[chroma > 0.5] = 1
+
+    # To compute the probability
+    occurProbs = np.empty(np.shape(count_pitch)[0])
+    
+    for i in range(np.shape(count_pitch)[0]) :
+        total = np.sum(count_pitch)
+        occurProbs[i] = np.sum(count_pitch[i]) / total
+
+    result = np.vstack((notes, np.round(occurProbs, 4))).T
+
+    ticks = range(12)
+    fig, ax = plt.subplots()
+    plt.title("Pitch Class")
+    plt.bar(ticks,occurProbs * 100, align='center')
+    plt.xticks(ticks, notes)
+    plt.xlabel("Note")
+    plt.ylabel("Number of occurrences %")
+
+    return fig, ax, result
+    
+    
+def pitch_class_histogram_chroma(y: npt.ArrayLike, sr: int, higher_resolution: bool, save_to_csv: bool = False) -> None :
+
+    S = np.abs(librosa.stft(y))
+    notes = np.array(librosa.key_to_notes('C:maj')) # For x-axis legend
+
+    if not higher_resolution :
+
+        chroma = librosa.feature.chroma_stft(S=S, sr=sr)
+        valid_pitch = np.empty(np.shape(chroma)) # To count pitch
+        valid_pitch[chroma < 0.7] = 0
+        valid_pitch[chroma >= 0.7] = 1
+        total = np.sum(valid_pitch)
+
+        # To compute the probability
+        # WARNING: (12,) means pure 1-D array
+        occurProbs = np.empty((12,))
+        for i in range(0, 12) :
+            occurProbs[i] = np.sum(valid_pitch[i]) / total
+
+        ticks = range(12)
+        colors = ['lightcoral', 'goldenrod', 'lightseagreen', 'indigo', 'lightcoral', 
+                 'goldenrod', 'lightseagreen', 'indigo', 'lightcoral', 'goldenrod', 
+                 'lightseagreen', 'indigo']
+        xLegend = notes
+
+        fig, ax = plt.subplots()
+        ax.bar(ticks,occurProbs * 100, align='center', color=colors)
+        ax.set_xticks(ticks)
+        ax.set_xticklabels(xLegend)
+        ax.set_title("Pitch Class Histogram")
+        ax.set_xlabel("Note")
+        ax.set_ylabel("Occurrences %")
+    
+    if higher_resolution :
+
+        chroma = librosa.feature.chroma_stft(S=S, sr=sr, n_chroma=120)
+        valid_pitch = np.empty(np.shape(chroma)) # To count pitch
+        valid_pitch[chroma < 0.7] = 0
+        valid_pitch[chroma >= 0.7] = 1
+        total = np.sum(valid_pitch)
+
+        occurProbs = np.empty((120,))
+        for i in range(0, 120) :
+            occurProbs[i] = np.sum(valid_pitch[i]) / total
+        
+        ticks = range(120)
+        fig, ax = plt.subplots()
+        xLegend = list()
+        for i in range(120) :
+            if i % 10 == 0 :
+                xLegend.append(notes[i // 10])
+            else :
+                xLegend.append('')
+
+        colors = list()
+        
+        for i in range(120) :
+            if i % 40 >=0 and i % 40 < 10 : colors.append('lightcoral')
+            elif i % 40 >= 10 and i % 40 < 20 : colors.append('goldenrod')
+            elif i % 40 >= 10 and i % 40 < 30 : colors.append('lightseagreen')
+            elif i % 40 >= 10 and i % 40 < 40 : colors.append('indigo')
+
+        fig, ax = plt.subplots()
+        ax.bar(ticks,occurProbs * 100, align='center', color = colors)
+        ax.set_xticks(ticks)
+        ax.set_xticklabels(xLegend)
+        ax.set_title("Pitch Class Histogram")
+        ax.set_xlabel("Note")
+        ax.set_ylabel("Occurrence %")
+
+    result = np.vstack((xLegend, np.round(occurProbs, 4))).T
+    if save_to_csv :
+        with open('pitch_class.csv', 'w') as out :
+            for row in result :
+                print(*row, sep=',', file=out) 
+                
+    return fig, ax, result

src/st_helper.py

@@ -0,0 +1,33 @@
+import streamlit as st
+
+@st.experimental_memo
+def convert_df(df):
+    """
+        Convert a pandas dataframe into a csv file.
+        For the download button in streamlit.
+    """
+    return df.to_csv(index=False).encode('utf-8')
+
+def show_readme(filename):
+    with st.expander("頁面說明(Page Description)"):
+        with open(filename, "r", encoding="utf-8") as f:
+            st.markdown(f.read())
+            
+            
+def get_shift(start_time, end_time):
+    """
+        回傳從start_time到end_time的時間刻度
+        開頭為start_time，結尾為end_time
+        中間每隔1秒一個刻度
+        
+        return: a np.array of time stamps
+    """
+    import numpy as np
+    
+    shift_array = np.arange(start_time, end_time, 1)
+    if shift_array[-1] != end_time:
+        shift_array = np.append(shift_array, end_time)
+    
+    shift_array = np.round(shift_array, 1)
+    return start_time, shift_array
+    

src/structure_analysis.py

@@ -0,0 +1,248 @@
+import numpy as np
+import os, sys, librosa
+from scipy import signal
+from matplotlib import pyplot as plt
+import matplotlib
+import matplotlib.gridspec as gridspec
+import IPython.display as ipd
+import pandas as pd
+from numba import jit
+
+import libfmp.b
+import libfmp.c2
+import libfmp.c3
+import libfmp.c4
+import libfmp.c6
+import libfmp
+from libfmp.b import FloatingBox
+
+from numpy import typing as npt
+import typing
+
+@jit(nopython=True)
+def compute_sm_dot(X, Y):
+    """Computes similarty matrix from feature sequences using dot (inner) product
+
+    Notebook: C4/C4S2_SSM.ipynb
+
+    Args:
+        X (np.ndarray): First sequence
+        Y (np.ndarray): Second Sequence
+
+    Returns:
+        S (float): Dot product
+    """
+    S = np.dot(np.transpose(X), Y)
+    return S
+
+def plot_feature_ssm(X, Fs_X, S, Fs_S, ann, duration, color_ann=None,
+                     title='', label='Time (seconds)', time=True,
+                     figsize=(5, 6), fontsize=10, clim_X=None, clim=None):
+    """Plot SSM along with feature representation and annotations (standard setting is time in seconds)
+
+    Notebook: C4/C4S2_SSM.ipynb
+
+    Args:
+        X: Feature representation
+        Fs_X: Feature rate of ``X``
+        S: Similarity matrix (SM)
+        Fs_S: Feature rate of ``S``
+        ann: Annotaions
+        duration: Duration
+        color_ann: Color annotations (see :func:`libfmp.b.b_plot.plot_segments`) (Default value = None)
+        title: Figure title (Default value = '')
+        label: Label for time axes (Default value = 'Time (seconds)')
+        time: Display time axis ticks or not (Default value = True)
+        figsize: Figure size (Default value = (5, 6))
+        fontsize: Font size (Default value = 10)
+        clim_X: Color limits for matrix X (Default value = None)
+        clim: Color limits for matrix ``S`` (Default value = None)
+
+    Returns:
+        fig: Handle for figure
+        ax: Handle for axes
+    """
+    cmap = libfmp.b.compressed_gray_cmap(alpha=-10)
+    fig, ax = plt.subplots(3, 3, gridspec_kw={'width_ratios': [0.1, 1, 0.05],
+                                              'wspace': 0.2,
+                                              'height_ratios': [0.3, 1, 0.1]},
+                           figsize=figsize)
+    libfmp.b.plot_matrix(X, Fs=Fs_X, ax=[ax[0, 1], ax[0, 2]], clim=clim_X,
+                         xlabel='', ylabel='', title=title)
+    ax[0, 0].axis('off')
+    libfmp.b.plot_matrix(S, Fs=Fs_S, ax=[ax[1, 1], ax[1, 2]], cmap=cmap, clim=clim,
+                         title='', xlabel='', ylabel='', colorbar=True)
+    ax[1, 1].set_xticks([])
+    ax[1, 1].set_yticks([])
+    libfmp.b.plot_segments(ann, ax=ax[2, 1], time_axis=time, fontsize=fontsize,
+                           colors=color_ann,
+                           time_label=label, time_max=duration*Fs_X)
+    ax[2, 2].axis('off')
+    ax[2, 0].axis('off')
+    libfmp.b.plot_segments(ann, ax=ax[1, 0], time_axis=time, fontsize=fontsize,
+                           direction='vertical', colors=color_ann,
+                           time_label=label, time_max=duration*Fs_X)
+    return fig, ax
+
+def SSM_chorma(wav_fn:str, anno_fn: str, hop_size: int = 4096, Nfft: int = 1024) -> None :
+    
+    x, fs = librosa.load(wav_fn)
+    duration= (x.shape[0])/fs
+
+    chromagram = librosa.feature.chroma_stft(y=x, sr=fs, tuning=0, norm=2, hop_length=hop_size, n_fft=Nfft)
+    X, Fs_X = libfmp.c3.smooth_downsample_feature_sequence(chromagram, fs/hop_size, filt_len=41, down_sampling=10)
+
+    # According to the documentation
+    ann, color_ann = libfmp.c4.read_structure_annotation(os.path.join(anno_fn), fn_ann_color=anno_fn)
+    ann_frames = libfmp.c4.convert_structure_annotation(ann, Fs=Fs_X) 
+    
+    X = libfmp.c3.normalize_feature_sequence(X, norm='2', threshold=0.001)
+    S = compute_sm_dot(X,X)
+    fig, ax = plot_feature_ssm(X, 1, S, 1, ann_frames, duration*Fs_X, color_ann=color_ann,
+                               clim_X=[0,1], clim=[0,1], label='Time (frames)',
+                               title='Chroma feature (Fs=%0.2f)'%Fs_X)
+    return fig, ax
+
+def plot_self_similarity(y_ref: npt.ArrayLike, sr: int, affinity: bool = False, hop_length: int = 1024) -> None:
+    '''
+    To visualize the similarity matrix of the signal
+
+    y_ref: reference signal
+    y_comp: signal to be compared
+    sr: sampling rate
+    affinity: to use affinity or not
+    hop_size
+    '''
+
+
+    # Pre-processing stage
+    chroma = librosa.feature.chroma_cqt(y=y_ref, sr=sr, hop_length=hop_length)
+    chroma_stack = librosa.feature.stack_memory(chroma, n_steps=10, delay=3)
+
+    fig, ax = plt.subplots()
+
+    if not affinity:
+        R = librosa.segment.recurrence_matrix(chroma_stack, k=5)
+        imgsim = librosa.display.specshow(R, x_axis='s', y_axis='s',
+                                        hop_length=hop_length)
+        plt.title('Binary recurrence (symmetric)')
+        plt.colorbar()
+
+    else:
+        R_aff = librosa.segment.recurrence_matrix(chroma_stack, metric='cosine',mode='affinity')
+        imgaff = librosa.display.specshow(R_aff, x_axis='s', y_axis='s',
+                                        cmap='magma_r', hop_length=hop_length)
+        plt.title('Affinity recurrence')
+        plt.colorbar()
+
+    return fig, ax
+
+@jit(nopython=True)
+def compute_kernel_checkerboard_gaussian(L: int =10 , var: float = 0.5, normalize=True) -> npt.ArrayLike:
+    """Compute Guassian-like checkerboard kernel [FMP, Section 4.4.1].
+    See also: https://scipython.com/blog/visualizing-the-bivariate-gaussian-distribution/
+
+    Notebook: C4/C4S4_NoveltySegmentation.ipynb
+
+    Args:
+        L (int): Parameter specifying the kernel size M=2*L+1
+        var (float): Variance parameter determing the tapering (epsilon) (Default value = 1.0)
+        normalize (bool): Normalize kernel (Default value = True)
+
+    Returns:
+        kernel (np.ndarray): Kernel matrix of size M x M
+    """
+    taper = np.sqrt(1/2) / (L * var)
+    axis = np.arange(-L, L+1)
+    gaussian1D = np.exp(-taper**2 * (axis**2))
+    gaussian2D = np.outer(gaussian1D, gaussian1D)
+    kernel_box = np.outer(np.sign(axis), np.sign(axis))
+    kernel = kernel_box * gaussian2D
+    if normalize:
+        kernel = kernel / np.sum(np.abs(kernel))
+    return kernel
+
+
+def compute_novelty_ssm(S, kernel: npt.ArrayLike = None, L: int = 10, var: float = 0.5, exclude: bool =False) -> npt.ArrayLike:
+    """Compute novelty function from SSM [FMP, Section 4.4.1]
+
+    Notebook: C4/C4S4_NoveltySegmentation.ipynb
+
+    Args:
+        S (np.ndarray): SSM
+        kernel (np.ndarray): Checkerboard kernel (if kernel==None, it will be computed) (Default value = None)
+        L (int): Parameter specifying the kernel size M=2*L+1 (Default value = 10)
+        var (float): Variance parameter determing the tapering (epsilon) (Default value = 0.5)
+        exclude (bool): Sets the first L and last L values of novelty function to zero (Default value = False)
+
+    Returns:
+        nov (np.ndarray): Novelty function
+    """
+    if kernel is None:
+        kernel = compute_kernel_checkerboard_gaussian(L=L, var=var)
+    N = S.shape[0]
+    M = 2*L + 1
+    nov = np.zeros(N)
+    # np.pad does not work with numba/jit
+    S_padded = np.pad(S, L, mode='constant')
+
+    for n in range(N):
+        # Does not work with numba/jit
+        nov[n] = np.sum(S_padded[n:n+M, n:n+M] * kernel)
+    if exclude:
+        right = np.min([L, N])
+        left = np.max([0, N-L])
+        nov[0:right] = 0
+        nov[left:N] = 0
+
+    return nov
+
+def SSM_Novelty(wav_filename:str, anno_csv_filename: str) -> None :
+
+    float_box = libfmp.b.FloatingBox()
+
+    fn_wav = os.path.join(wav_filename)
+    ann, color_ann = libfmp.c4.read_structure_annotation(os.path.join(anno_csv_filename), 
+                                                         fn_ann_color=anno_csv_filename)
+
+    S_dict = {}
+    Fs_dict = {}
+    x, x_duration, X, Fs_X, S, I = libfmp.c4.compute_sm_from_filename(fn_wav, 
+                                                    L=11, H=5, L_smooth=1, thresh=1)
+
+    S_dict[0], Fs_dict[0] = S, Fs_X
+    ann_frames = libfmp.c4.convert_structure_annotation(ann, Fs=Fs_X) 
+    fig, ax = libfmp.c4.plot_feature_ssm(X, 1, S, 1, ann_frames, x_duration*Fs_X,
+                label='Time (frames)', color_ann=color_ann, clim_X=[0,1], clim=[0,1], 
+                title='Feature rate: %0.0f Hz'%(Fs_X), figsize=(4.5, 5.5))
+    float_box.add_fig(fig)
+
+    x, x_duration, X, Fs_X, S, I = libfmp.c4.compute_sm_from_filename(fn_wav, 
+                                                    L=41, H=10, L_smooth=1, thresh=1)
+    S_dict[1], Fs_dict[1] = S, Fs_X
+    ann_frames = libfmp.c4.convert_structure_annotation(ann, Fs=Fs_X) 
+    fig, ax = libfmp.c4.plot_feature_ssm(X, 1, S, 1, ann_frames, x_duration*Fs_X,
+                label='Time (frames)', color_ann=color_ann, clim_X=[0,1], clim=[0,1], 
+                title='Feature rate: %0.0f Hz'%(Fs_X), figsize=(4.5, 5.5))
+    float_box.add_fig(fig)
+    float_box.show()
+
+    figsize=(10,6)
+    L_kernel_set = [5, 10, 20, 40]
+    num_kernel = len(L_kernel_set)
+    num_SSM = len(S_dict)
+
+    fig, ax = plt.subplots(num_kernel, num_SSM, figsize=figsize)
+    for s in range(num_SSM):
+        for t in range(num_kernel):
+            L_kernel = L_kernel_set[t]
+            S = S_dict[s]
+            nov = compute_novelty_ssm(S, L=L_kernel, exclude=True)        
+            fig_nov, ax_nov, line_nov = libfmp.b.plot_signal(nov, Fs = Fs_dict[s], 
+                    color='k', ax=ax[t,s], figsize=figsize, 
+                    title='Feature rate = %0.0f Hz, $L_\mathrm{kernel}$ = %d'%(Fs_dict[s],L_kernel)) 
+            libfmp.b.plot_segments_overlay(ann, ax=ax_nov, colors=color_ann, alpha=0.1, 
+                                           edgecolor='k', print_labels=False)
+    plt.tight_layout()
+    plt.show()  
+    

src/timbre_analysis.py

@@ -0,0 +1,125 @@
+import librosa
+from librosa import display
+from librosa import feature
+
+import numpy as np
+from numpy import typing as npt
+
+from matplotlib import pyplot as plt
+import scipy
+
+
+def spectral_centroid_analysis(y: npt.ArrayLike, sr: int, shift_array: npt.ArrayLike) -> None :
+
+    S, phase = librosa.magphase(librosa.stft(y=y))
+    cent = librosa.feature.spectral_centroid(S=S)
+    times = librosa.times_like(cent, sr)
+
+    fig, ax = plt.subplots()
+    librosa.display.specshow(librosa.amplitude_to_db(S, ref=np.max),
+                             y_axis='log', x_axis='time', ax=ax, sr=sr)
+    ax.plot(times, cent.T, label='Spectral centroid', color='w')
+    ax.legend(loc='upper right')
+    ax.set(title='log Power spectrogram')
+    ax.set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax.autoscale()
+
+    result = np.vstack((times, cent))
+
+    return fig, ax, result
+
+
+def rolloff_frequency_analysis(y: npt.ArrayLike, sr: int, roll_percent:float = 0.99,
+                               shift_array: npt.ArrayLike =None) -> None :
+
+    rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr, roll_percent=roll_percent)
+    rolloff_min = librosa.feature.spectral_rolloff(y=y, sr=sr, roll_percent=0.01)
+    times = librosa.times_like(rolloff, sr)
+    S, phase = librosa.magphase(librosa.stft(y))
+
+    fig, ax = plt.subplots()
+    librosa.display.specshow(librosa.amplitude_to_db(S, ref=np.max),
+                             y_axis='log', x_axis='time', ax=ax, sr=sr)
+    ax.plot(librosa.times_like(rolloff,sr), rolloff[0], label=f'Roll-off frequency ({roll_percent})')
+    ax.plot(librosa.times_like(rolloff,sr), rolloff_min[0], color='w',
+            label='Roll-off frequency (0.01)')
+    ax.legend(loc='lower right')
+    ax.set(title='log Power spectrogram')
+    ax.set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax.autoscale()
+
+    result = np.vstack((times, rolloff, rolloff_min))
+
+    return fig, ax, result
+
+def spectral_bandwidth_analysis(y: npt.ArrayLike, sr: int, shift_array: npt.ArrayLike =None) -> None :
+    
+    S, phase = librosa.magphase(librosa.stft(y=y))
+    spec_bw = librosa.feature.spectral_bandwidth(S=S)
+    times = librosa.times_like(spec_bw, sr)
+
+    fig, ax = plt.subplots(nrows=2, sharex=True)
+    centroid = librosa.feature.spectral_centroid(S=S, sr=sr)
+    ax[0].semilogy(times, spec_bw[0], label='Spectral bandwidth')
+    ax[0].set(ylabel='Hz', xticks=[], xlim=[times.min(), times.max()])
+    ax[0].legend()
+    ax[0].label_outer()
+    librosa.display.specshow(librosa.amplitude_to_db(S, ref=np.max),
+                             y_axis='log', x_axis='time', ax=ax[1], sr=sr)
+    ax[1].set(title='log Power spectrogram')
+    ax[1].fill_between(times, np.maximum(0, centroid[0] - spec_bw[0]),
+                       np.minimum(centroid[0] + spec_bw[0], sr/2),
+                       alpha=0.5, label='Centroid +- bandwidth')
+    ax[1].plot(times, centroid[0], label='Spectral centroid', color='w')
+    ax[1].legend(loc='lower right')
+    ax[1].set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax[1].autoscale()
+
+    result = np.vstack((times, spec_bw))
+
+    return fig, ax, result
+
+
+def harmonic_percussive_source_separation(y: npt.ArrayLike, sr: int,
+        shift_array: npt.ArrayLike =None                                      
+    ) -> None :
+
+    D = librosa.stft(y)
+    H, P = librosa.decompose.hpss(D)
+    t = librosa.frames_to_time(np.arange(D.shape[1]), sr=sr)
+    
+    fig, ax = plt.subplots(nrows=3, sharex=False, sharey=False, figsize=(12, 8))
+    # 設置子圖之間的水平間距和垂直間距
+    plt.subplots_adjust(hspace=0.6, wspace=0.3)
+    img = librosa.display.specshow(librosa.amplitude_to_db(np.abs(D),ref=np.max),
+                                   y_axis='log', x_axis='time', ax=ax[0], sr=sr)
+    ax[0].set(title='Full power spectrogram')
+    #// ax[0].label_outer()
+    ax[0].set_xlabel('') # 不顯示x軸名稱
+    ax[0].set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax[0].autoscale()
+
+    librosa.display.specshow(librosa.amplitude_to_db(np.abs(H), ref=np.max(np.abs(D))),
+                             y_axis='log', x_axis='time', ax=ax[1], sr=sr)
+    ax[1].set(title='Harmonic power spectrogram')
+    #// ax[1].label_outer()
+    ax[1].set_xlabel('') # 不顯示x軸名稱
+    ax[1].set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax[1].autoscale()
+
+    librosa.display.specshow(librosa.amplitude_to_db(np.abs(P), ref=np.max(np.abs(D))),
+                             y_axis='log', x_axis='time', ax=ax[2], sr=sr)
+    ax[2].set(title='Percussive power spectrogram')
+    ax[2].set_xticks(shift_array - shift_array[0],
+                         shift_array)
+    ax[2].autoscale()
+    
+    fig.colorbar(img, ax=ax, format='%+2.0f dB')
+
+    return fig, ax, (D, H, P, t)
+