Skip to main content

OpenAI Learning:- Chapter 5

 Generative AI-Powered Audio/Video Processing: Whisper's Python Adventure

What Is the Purpose of This Application?
An application that displays the ability of Generative AI to process and analyze audio/video files, and then output the lyrics for that audio or video file.

Sample code:

import streamlit as st
from pytube import YouTube
import os
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

def get_mp3(url):
    yt = YouTube(str(url))
    audio = yt.streams.filter(only_audio = True).first()
    destination = '.'
    out_file = audio.download(output_path=destination)
    base, ext = os.path.splitext(out_file)
    new_file = base + '.mp3'
    os.rename(out_file, new_file)
    return new_file

def get_transcript(audio_file):
    device = "cuda:0" if torch.cuda.is_available() else "cpu"  #If you have GPU else it will use cpu
    torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
    model_id = "openai/whisper-tiny"    # define the model
    model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True)
    model.to(device)
    processor = AutoProcessor.from_pretrained(model_id)
   
    # create the pipeline
    pipe = pipeline(
        "automatic-speech-recognition",
        model=model,
        tokenizer=processor.tokenizer,
        feature_extractor=processor.feature_extractor,
        max_new_tokens=128,
        chunk_length_s=30,
        batch_size=16,
        return_timestamps=True,
        torch_dtype=torch_dtype,
        device=device,
        generate_kwargs={"language": "english"})
   
    result = pipe(audio_file, return_timestamps=True,generate_kwargs={"language": "english"})
    result = result["chunks"]
    return result

def format_lyrics(lyrics):
    formatted_lyrics = ""
    for line in lyrics:
        text = line["text"]
        formatted_lyrics += f"{text}\n\n"
    return formatted_lyrics.strip()

def fetch_lyrics(url):
    mp3 = get_mp3(url)
    status_placeholder = st.empty()
    status_placeholder.subheader("Please wait for few seconds. Preparing the lyrics for you...")
    lyrics = get_transcript(mp3)
    status_placeholder.empty()
    lyrics = format_lyrics(lyrics)
    return lyrics

def main():
    text_color = "#7dd100"
    #st.markdown('<p style="color: #006fd1; font-family: sans-serif; text-align:center; font-size: 65px;"><b>YouTube Lyrics App</b></p>', unsafe_allow_html=True)
    st.markdown("""<p style="color: #d15000;font-size: 70px;font-family: sans-serif; text-align:center;margin-bottom:0px;"><b>Lyrics</b><span style="color: #E94B3CFF;font-size: 70px;font-family: sans-serif;"><b>Box</b></span></p>""", unsafe_allow_html=True)
    st.markdown('<p style="font-family: sans-serif; text-align:center; font-size: 20px; margin-bottom:60px;">Get the Lyrics of your Favorite Song for Free</p>', unsafe_allow_html=True)
    # Input field for the user to enter the URL of the song
    st.markdown('<p style="font-family: sans-serif; text-align:left; font-size: 20px; margin-bottom:0px;">Enter the audio/video link below:</p>', unsafe_allow_html=True)
    url = st.text_input("", "")

    if url:
        # Button to trigger fetching and displaying the lyrics
        if st.button("Get Lyrics of this A/V"):
            lyrics = fetch_lyrics(url)
            st.subheader("Lyrics:")
            st.write(lyrics)

    st.markdown('<p style="font-size: 35px;font-family: sans-serif; text-align:left; margin-top: 100px;"><b>How to Get the Lyrics of your Video?</b></p>', unsafe_allow_html=True)
    st.markdown('<p style="font-family: sans-serif; text-align:left; font-size: 20px">To extract the lyrics of your favorite video using this tool follow the steps. <br /> <br /> &ensp; 1. Copy the link of the video from Youtube. \
                <br /> &ensp; 2. Paste the link in the box above. <br /> &ensp; 3. Hit the "Get Lyrics of this A/V" button. </p>', unsafe_allow_html=True)
   
    st.markdown('<p style="font-size: 35px;font-family: sans-serif; text-align:left; margin-top: 40px;"><b>Why Should you use this tool?</b></p>', unsafe_allow_html=True)
    st.markdown('<p style="font-family: sans-serif; text-align:left; font-size: 20px">Features of this tool are given below: <br /> <br /> \
                &ensp; 1. This Tool uses <a href="https://huggingface.co/openai/whisper-tiny/">OpenAI Whisper</a> to extract transcript from audio file.\
                <br /> &ensp; 2. We do not save your data or video.\
                <br /> &ensp; 3. Easy and Free-to-use.</p>', unsafe_allow_html=True)

if __name__ == "__main__":
    main()
Labels:

Comments

Post a Comment

Popular posts from this blog

Clipboardy: Node.js library

 Clipboardy is a popular Node.js library that provides a cross-platform solution for interacting with the system clipboard. It supports both synchronous and asynchronous operations, and it can be used to copy, paste, and read the clipboard contents. Features of Clipboardy: 1. Cross-platform compatibility: It works on macOS, Windows, Linux, OpenBSD, FreeBSD, Android with Termux, and modern browsers. 2. Synchronous and asynchronous operations: It provides both synchronous and asynchronous methods for clipboard operations, making it suitable for a variety of use cases. 3. Copy, paste, and read operations: It supports copying, pasting, and reading the clipboard contents, making it a versatile tool for interacting with the clipboard. 4. Promise-based API: It is an asynchronous method that returns promises, making it easy to handle asynchronous operations in a clean and concise way. 5. Easy to use: It has a simple and intuitive API that is easy to learn and use. How to Use Clipboardy: To...
Post a Comment
Read more

ARIA Snapshot in Playwright

  What is an ARIA Snapshot in Playwright? An  ARIA snapshot  in Playwright is a structured representation of a page’s  accessibility tree , which is used by assistive technologies (e.g., screen readers) to interpret the content of a web page. This snapshot helps verify if elements have the correct  roles, names, and properties  required for accessibility. Playwright provides the page.accessibility.snapshot() API to capture this accessibility tree at any given moment during test execution. How Does ARIA Work? ARIA ( Accessible Rich Internet Applications ) is a set of attributes that help improve accessibility by defining roles, states, and properties for elements that are not natively accessible. Example: In this case, the aria-label ensures that screen readers identify the button as “Submit Form.” How to Use ARIA Snapshots in Playwright? Playwright’s  accessibility.snapshot()   method retrieves the  accessible structure  of the page. Ex...
Post a Comment
Read more

Bruno vs Postman: Which API Client Should You Choose?

  As API testing becomes more central to modern software development, the tools we use to test, automate, and debug APIs can make a big difference. For years, Postman has been the go-to API client for developers and testers alike. But now, Bruno , a relatively new open-source API client, is making waves in the community. Let’s break down how Bruno compares to Postman and why you might consider switching or using both depending on your use case. ✨ What is Bruno? Bruno is an open-source, Git-friendly API client built for developers and testers who prefer simplicity, speed, and local-first development. It stores your API collections as plain text in your repo, making it easy to version, review, and collaborate on API definitions. 🌟 What is Postman? Postman is a full-fledged API platform that offers everything from API testing, documentation, and automation to mock servers and monitoring. It comes with a polished UI, robust integration, and support for collaborati...
Post a Comment
Read more