Heart2Heart is shutting down after seven years.

Diff

Member

From: Middle of nowhere, Kansas

Joined: 2017-02-15

Posts: 658

Files: 139

PM

Heartbeat overlay

If any of you are interested in the heartbeat overlay I used on my two video game videos, well here it is. You need Python 3 installed, and once you have that you need to use pip to install PyAudio, Numpy, and PyQt5. Got all that done? Cool, all you need to do is run this script then:

# Original script by: Will Yager
# http://yager.io/LEDStrip/LED.html
# Mod by Different55 <burritosaur@protonmail.com>
# Seriously he did like 99% of the work here, all I changed was
# the way the script displays the info it processes.

import pyaudio as pa
import numpy as np
from math import pi, atan
from time import time
from collections import deque
import sys
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *

### Below is the file notes_scaled_nosaturation.py by Will Yager

# Will Yager
# This Python script sends color/brightness data based on
# ambient sound frequencies to the LEDs.

def fft(audio_stream):
	def real_fft(im):
		im = np.abs(np.fft.fft(im))
		re = im[0:len(im)/2]
		re[1:] += im[len(im)/2 + 1:][::-1]
		return re
	for l, r in audio_stream:
		yield real_fft(l) + real_fft(r)

def scale_samples(fft_stream, leds):
	for notes in fft_stream:
		yield notes[0:leds]

def rolling_smooth(array_stream, falloff):
	smooth = array_stream.__next__()
	yield smooth
	for array in array_stream:
		smooth *= falloff
		smooth += array * (1 - falloff)
		yield smooth

def add_white_noise(array_stream, amount):
	for array in array_stream:
		if sum(array) != 0:
			yield array + amount
		else:
			yield array

def rolling_scale_to_max(stream, falloff):
	avg_peak = 0.0
	for array in stream:
		peak = np.max(array)
		if peak > avg_peak:
			avg_peak = peak # Output never exceeds 1
		else:
			avg_peak *= falloff
			avg_peak += peak * (1-falloff)
		if avg_peak == 0:
			yield array
		else:
			yield array / avg_peak

# [[Float 0.0-1.0 x 32]]
def process(audio_stream, num_leds, num_samples, sample_rate, sensitivity):
	frequencies = [float(sample_rate*i)/num_samples for i in range(num_leds)]
	notes = fft(audio_stream)
	notes = scale_samples(notes, num_leds)
	notes = add_white_noise(notes, amount=2000)
	notes = rolling_scale_to_max(notes, falloff=.98) # Range: 0-1
	notes = rolling_smooth(notes, falloff=.1)
	return notes

### And that's all he wrote.

app = QApplication(sys.argv) # Our application

window = QWidget() # The window we'll be using.

window.setAttribute(Qt.WA_TranslucentBackground, True) # Allows us to have a transparent window.
window.setWindowFlags(Qt.SplashScreen | Qt.FramelessWindowHint | Qt.WindowStaysOnTopHint | Qt.WindowSystemMenuHint | Qt.WindowDoesNotAcceptFocus | Qt.CustomizeWindowHint | Qt.X11BypassWindowManagerHint | Qt.WindowTransparentForInput)
# Qt.SplashScreen identifies ourselves as a splashscreen-type loading window, which sometimes are allowed to hang out on top.
# Qt.FramelessWindowHint asks to be made free of all window chrome and decoration. No close buttons.
# Qt.WindowStaysOnTopHint asks to be allowed to stay on top of everything else.
# Qt.WindowDoesNotAcceptFocus just says that we want all the attention to be somewhere else.


# Create the button
ratetext = QLabel() # The label that holds the current heart rate.
ratetext.setText("Loading...") # Let's make it say "Loading" while we're loading.
ratetext.setStyleSheet('font-size: 20pt; font-weight: 100; color: rgba(200, 200, 200, 0.7);') # Some stylistic options.

shadow = QGraphicsDropShadowEffect() # This is a drop shadow effect that we'll use as a weak outline.
shadow.setColor(QColor(0, 0, 0, 200)) # Black shadow, slight transparency
shadow.setBlurRadius(3) # Slight blur to make it actually visible around the edges because...
shadow.setOffset(0, 0) # It's sitting directly below the text.
ratetext.setGraphicsEffect(shadow) # And apply the effect.

heart = QLabel() # The heart icon. 
icon = '♡' # I actually used an emoji for mine but I guess not everyone has emoji on their computers.
sheet = 'min-width: 20pt; font-weight: 600;' # The style.
heart.setText(icon) # Insert text
heart.setStyleSheet(sheet) # apply styling

hshadow = QGraphicsDropShadowEffect() # Let's apply that shadow to our heart as well.
hshadow.setColor(QColor(0, 0, 0, 200)) 
hshadow.setBlurRadius(3)
hshadow.setOffset(0, 0)
heart.setGraphicsEffect(hshadow)

line = QLabel() # This will contain the picture of that fancy line thing.

pic = QPicture() # This is the picture of that fancy line thing.

paint = QPainter(pic) # This is the misunderstood artist who will be painting that fancy line thing.
paint.setPen(QPen(Qt.white, 1, Qt.SolidLine, Qt.RoundCap)) # For a "loading" line, opaque white.
paint.setRenderHint(QPainter.Antialiasing) # Turn on Antialiasing.
paint.setRenderHint(QPainter.HighQualityAntialiasing) # "'HighQualityAntialiasing'? Sure that sounds nice, let's turn that on."
paint.drawLine(0, 25, 150, 25) # Flatlined.
paint.end() # And the painter is finished.

vbox = QVBoxLayout() # This is our main layout.
hbox = QHBoxLayout() # This is the "informational" layout on top with the rate and heart icon.

hbox.addWidget(ratetext) # Add the heart rate to the horizontal box.
hbox.addStretch(1) # Add a stretcher that will suck up as much space as it can.
hbox.addWidget(heart) # Add the heart icon to the horizonal box.
hbox.addStretch(1) # And another stretcher, make them fight each other. The point of this is to basically "center" the heart icon in the leftover space.

vbox.addLayout(hbox) # Add that horizontal box to the vertical box.
vbox.addWidget(line) # Add that fancy line thing to the vertical box.
window.setLayout(vbox) # And set the vertical box as our main layout here.

window.resize(150, 20) # Resize the window to 150x20.

window.move(0, 30) # Relocate it to the top left corner.

window.show() # Lights, camera, action.

# Function to open the audio stream.
audio_stream = pa.PyAudio().open(format=pa.paInt16, \
								channels=2, \
								rate=44100, \
								input=True, \
								# Uncomment and set this using find_input_devices.py
								# if default input device is not correct
								#input_device_index=2, \
								frames_per_buffer=1156)

# Convert the audio data to numbers, num_samples at a time.
def read_audio(audio_stream, num_samples):
	while True:
		# Read all the input data. 
		samples = audio_stream.read(num_samples) 
		# Convert input data to numbers
		samples = np.fromstring(samples, dtype=np.int16).astype(np.float)
		samples_l = samples[::2]  
		samples_r = samples[1::2]
		yield samples_l, samples_r

bpm = deque([-1]*20, maxlen=20) # This holds a history of BPM that's 20 beats long.
vol = deque([0]*50, maxlen=50) # This holds a history of the volume, we use it to make the line. It's 50... somethings long. "Frames" maybe.

audio = read_audio(audio_stream, num_samples=1156) # Open the audio
leds = process(audio, num_leds=32, num_samples=1156, sample_rate=44100, sensitivity=1.3) # Process the audio. Remember, this used to be a music visualizer script for LEDs.

beattime = time() # The timestamp of the current beat
last_beattime = time() # Timestamp of the last beat

bassing = False # Whether or not we are currently hearing a beat

for frame in leds:
	bass = np.max(frame[:3]) # That's one way to get a low pass filter I guess.
	
	if (vol[-1] > vol[-2] or vol[-1] == 1.0) and vol[-1] > bass and not bassing and vol[-1] > 0.2: # If we just saw the peak of a spike
		bassing = True # We're in the middle of a beat
		beattime = time() # Update the beat time
		bpm.append(60.0/(beattime-last_beattime)) # Tack on the time since the last beat. This might be wrong.
		last_beattime = beattime # Update the last beat time
	elif (vol[-1] < 0.1 and bassing and time()-beattime > 0.125): # If we've fallen back to the peace and quiet
		bassing = False # Stop bassing
	
	vol.append(bass) # Make a not of the current volume level.
	
	total = 0 # Sum of every BPM value we've seen recently
	calibrating = False # Whether or not we've seen some things.
	for beat in bpm: # Loop over all our BPMs.
		total = total + beat # Add it to the total.
		if beat < 0: # If it's -1 that means it's "empty" and the script has detected fewer than 20 heartbeats.
			calibrating = True # So let's not show the heart rate right now.
			break
			
	rate = (total/(len(bpm)*2)) # Calculate the heart rate.
	
	font = str(20-(max(0, bass-0.1))*6) # Calculate font size for the icon based on volume. Louder = smaller.
	color = bass*50 # Calculate color for the icon. Louder = redder.
	heart.setStyleSheet('font-size: '+font+'pt; color: rgba('+str(200+color)+', '+str(200-color)+', '+str(200-color)+', 0.7); '+sheet) # Put it all in a stylesheet.
	
	paint.begin(pic) # Hire our misunderstood artist again.
	paint.setRenderHint(QPainter.Antialiasing) # Turn these back on.
	paint.setRenderHint(QPainter.HighQualityAntialiasing)
	for i, v in enumerate(vol): # Flash our life before our eyes
		paint.setPen(QPen(QColor(0, 0, 0, 100), 2, Qt.SolidLine, Qt.RoundCap)) # Transparent black color to serve as an outline.
		paint.drawLine(i*3, 25-vol[i-1]*25, i*3+2, 25-v*25) # Draw the outline.
		color = 0 #max(0, v-0.1)*20 # I've thought about making the tips of spikes reddish but decided against it.
		paint.setPen(QPen(QColor(200+color, 200-color, 200-color, 178), 1.5, Qt.SolidLine, Qt.RoundCap)) # Our main transparent white line.
		paint.drawLine(i*3, 25-vol[i-1]*25, i*3+2, 25-v*25) # Draw it.
	paint.end() # Masterpiece has been created.
	line.setPicture(pic) # Show it off to the world.
		
	debug = False # Set to "true" for fancy terminal visualizer.
	if debug:
		if bassing: # If we're in the middle of a heartbeat, add some red stars.
			star = '\033[91m***\033[0m'
		else:
			star = ''
		print(str(bass).zfill(20), '#'*int(bass*50), star) # Print a bar based on the loudness.
	if calibrating: # If we're calibrating,
		ratetext.setText('???') # Don't show the BPM.
	else: # Otherwise
		ratetext.setText(str(round(rate, 1))) # Do.
	
	app.processEvents() # Tells the GUI bit of our app to wake up and OBEY.

Be warned: someone pointed out yesterday that it can get up to about 10BPM faster than reality. Not sure what's going on there.

Diff

Member

From: Middle of nowhere, Kansas

Joined: 2017-02-15

Posts: 658

Files: 139

PM

Re: Heartbeat overlay

Figured out the problem with this. The reason why it's about 10BPM faster than reality. Well. I say "I", I mean some dude figured it out. I don't completely understand how it messes up yet but my understanding of it right now is that I mixed up units. I store BPM in a list and average it together wrong. When I try to average it together, I divide it by number of beats when apparently I should be dividing it by amount of time that's passed between the oldest and newest beats. This is very wrong, but only wrong enough that I'm only ever off by about 10BPM.

Anyway, going to be rewriting that bit to fix the problem and then I should be getting a few more videos uploaded. Probably not this week though. My work schedule's pretty busy. Probably not next week either, I don't have my schedule for next week yet but there's a posted notice that we can't ask for time off for 2 of the days next week, which is probably going to upset the rest of the week, too.