A Visit to eSight

After working on the All-Seeing Pi, a series of events unfolded such that Dan and I were put in contact with a company called eSight. The founder of eSight had the same inspiration as the All-Seeing Pi many years ago, and has now developed the third generation of a vision assist platform.

So we were invited to come check out their technology, and see how it stacks up to our contraption. For starters, it is a very sleek and lightweight design that looks a little more stylish. It also has some great vision enhancement functionality like contrast boosting, magnification, and screenshots. Using these features, Dan was able to read an eye chart at the "20-30" level, which is better than he was able to see  before he lost his vision!

This hasn't stopped our interest in the All-Seeing Pi though, as eSight comes with a pretty hefty price tag of $10k. An existing commercial solution is never a good reason to stop a DIY project too!

The All-Seeing Pi

This post is a about vision enhancement platform called The All-Seeing Pi that I have been working on with my friend Dan, who is blind. People who are blind rarely have no vision at all though, and in Dan's case, he still has a little bit of sight in one eye. He's also the first to tell you how much technology can do to enable mobility.

From these discussions, we came up with the idea for a video feed connected to a display, with a wearable screen in the ideal spot for maximum vision. This allows someone to focus on just the screen, and let the camera capture the detail and depth of the environment.

In the end, the prototype served as a successful proof of concept. Checkout the video above for a field test and some more discussion! Dan also likes to push the limits of what can be done with his disability, which he chronicles at his blog Three Points of Contact.

In the rest of this post, I'll be talking about how to build the device. This may be useful if you or a friend have a similar condition, but it is also a great starting platform for a Raspberry Pi based augmented reality rig. The general setup is a raspberry pi with a camera module running on an HDMI (not SPI!) small display. The video feed is provided via OpenCV and RaspiCam, with code and install details below.

Video Tutorial: Astrophotography Barn Door Tracker

This is simply a video overview of my previous post on astrophotography barn door trackers. After lots of questions about how to use it, I thought a video might come in handy!

N-Body Orbit Simulation with Runge-Kutta

In a previous post I introduced a simple orbital simulation program written in python. In that post we simulated orbits by simply taking the location, and velocities of a set of masses, and computed the force on each body. Then we calculated where all the bodies would end up under that force a small time step into the future. This process was repeated over and over again, and was used to simulate gravitational systems like our solar system giving outputs like you see below.

This technique is called the Euler method. If you're not familiar with using numerical methods to simulate orbits, I'd recommend taking a look there first!

Part 1: Python N Body Simulation

Orbit paths from the previous example

In this post I will be adding a more advanced time stepping technique called the Fourth Order Runge-Kutta method. Kids these days just call it RK4. I'll walk through the logic behind RK4, and share a python implementation. I will also link to a C++ implementation, and do a brief performance comparison.

Fourth Order Runge-Kutta

Launch a Script Using Alexa Voice Commands

In a previous post, I showed how you can build a smart mirror with an Alexa voice assistant on board. The MagicMirror software and Alexa voice assistant were both hosted on a Raspberry Pi, but unfortunately there was no obvious way to get Alexa to control the smart mirror, or deliver commands to the Raspberry Pi.

I have now found a solution that is free, reliable, and very flexible. This is done by writing an Alexa Skill that adds a message to a cloud hosted queue based on your voice command. The Raspberry Pi repeatedly checks this queue for new messages, and runs customizable behaviour based on message contents. This is not limited to smart mirror applications, or Raspberry Pis. It can be used to launch any script you want on any platform that will connect to Amazon's SQS.

Here is a demonstration, and high level overview of how it works:

and a follow up demonstrating an extension of this idea:

In this tutorial I will focus on just using this to simply turn the smart mirror on and off. Adding your own scripts should then be fairly straight forward,

The steps will be as follows:

1. Create a Queue using the Amazon Simple Queue Service (SQS)
2. Write some python code to read and write to this queue
3. Write a Lambda Function that posts messages to the queue
4. Write an Alexa Skill that calls the Lambda Function
5. Schedule a task on your Raspberry Pi to read queue messages and take appropriate action.

DIY Selfie Stick

This is a fun and very easy project I came up with while trying to build a wireless shutter for an iPhone. I knew selfie sticks could trigger your camera shutter, so I was trying to find out what information they were sending through the 3.5mm aux port (aka headphone plug) to do this.

In this process I learned that headphone volume buttons will also trigger the camera shutter!

So I plugged in some headphones with volume control, and tried to come up with a way to attach my phone to a stick. I came up with the spatula sponge elastic combo you'll see here.

Take a look for yourselves!

Markov Chains: The Imitation Game

In this post we're going to build a Markov Chain to generate some realistic sounding sentences impersonating a source text. This is one of my favourite computer science examples because the concept is so absurdly simple and and the payoff is large. This will be done using python, and your final code will look like this.

Before I explain how it works though, let's look at an example generated from the past four Cyber Omelette posts:

"The first step to attach our mirror is done by simply screw the backing with a Raspberry Pi's into the filename. That looks like installing and blinking cursor. Type the wood screws.

Gorilla glue.

Electrical tape.

Extension Cord with multiple passes of the all Together  Once it hasn't really stuck in place until all directions. Clean your monitor." 

So it's not exactly coherent, but it comes pretty close! I also built a twitter bot called mayhem_bot (git repo) using this concept. This bot imitates users and trending hashtag discussions, so send it some tweets if you want more examples.

Rat dreams of stealing the treat bag and NOT getting caught. #unlikelyanimaldreams #mayhem_bot

— Mayhem Bot (@mayhem_bot) December 20, 2016

Please join us Friday at the live pic.twitter.com/ZTHpH5WgoS Human? #cbcharbourlights #mayhem_bot

— Mayhem Bot (@mayhem_bot) December 9, 2016

DIY Smart Mirror ft. Alexa

Materials

1. Computer Monitor. Buy used and look for cable inputs that point down or sideways instead of straight out. 22" recommended (I used Acer AL2216 )
2. Video cable. Whatever fits your monitor (I used HDMI to DVI adapter)
3. Raspberry Pi 3, 16GB Micro SD Card, 5V Micro USB power supply, any RPi case
4. Wood. I bought four 3ft lengths of 3"x1"
5. 8 Angle brackets
6. Short wood screws.
7. Gorilla glue.
8. Electrical tape.
9. Extension Cord with multiple plugs.
10. 50cm x 70cm Ikea STROMBY frame.
11. Gila Mirrored Window Film 
12. USB Sound Card, Lapel Mic, any USB powered speaker *

*Bonus Alexa Materials

Additional required tools/supplies are: scotch tape, a squirt bottle, dish soap, a drill or screwdriver, exacto knife, scissors, and a saw.

March 10, 2018 update: I'm currently in the process of trying out an actual two way mirror as well instead of the window film. If you're interested in this I can recommend this mirror which comes in several sizes. I'll have a post coming soon with instructions for how to mount it.

Creating the Two Way Mirror

The two way mirror is the most essential part of this build. To create this component, you just apply Gila Mirrored Window Film to any piece of glass or acrylic. This works equally well for glass and acrylic, and only takes about 15 minutes.