Now that we have a function for visualizing the surroundings and one for listening to the surroundings we can put them together. The trick here is to have both run at the same time. The simple way to do this is to use threads. Thus the program startup program looks like this:
from visual_cortex import lookAtSurroundings from audio_cortex import listenToSurroundings import thread try: thread.start_new_thread( listenToSurroundings,("AudioCortex",)) thread.start_new_thread( lookAtSurroundings,("VisualCortex",)) except: print "Error, unable to start thread." while 1: pass
This program simply loads the respective stimuli as threads (here I put
visual_cortex.py and put
audio_cortex.py). This allows both programs to run simultaneously.
Important note: Theoretically speaking the above is all you will have to do. However, there are some small issues I ran into. My Raspberry Pi had trouble taking pictures and taking audio at the exact same time (a memory issue, I believe, but I’m unsure). To fix this, I made a global file
config.py that is imported into each module and contains global flags that will be set whenever audio/images are being acquired. This way, before taking audio/images it can check the flags and wait until they’ve been reset. I will explain this more in detail, but feel free to check it out exactly on my github.