TCS34725 and ESP8266 based Color Sensor Glove

Color sensing glove built with TCS34725, ESP8266 and Pro Trinket

In the earlier posts here, here and here, I was leading up to this build: The Color Catcher and Thrower. A Pro Trinket based glove that captures colors with TCS34725 and sends it out as MQTT message through ESP8266 for my WiFi Christmas Tree and other light fixtures to consume.

The MQTT messages sent during the demo:

MQTT message from LightGlove
MQTT message from LightGlove

 

Hardware Components

  1. Adafruit Pro Trinket (5v)
  2. ESP8266
  3. Adafruit TCS34725 5v Color sensor
  4. Flex Sensor
  5. RGB Led
  6. Resistors
  7. 28 Pin IC socket for Trinket
  8. 2x4Pin female headers as socket for ESP8266
  9. 5v usb battery charger (this one)

On the receiving end:

  1. Arduino Yun
  2. Infineon RGB Led Shield
  3. RGB Led Strip from Amazon

About TCS34725 Color Sensor

TCS34725 is a color light to digital converter with IR filter developed by Texas Advanced OptoElectronic Solutions (Now bought by AMS). This contains a 3 x 4 photo diode array composed of red, green, blue-filtered photo diodes and a clear unfiltered photo diode. The ADC’s simulatenously convert the amplified photo diode currents to a 16-bit digital value and transfer them to data registers. This data can be read over the I2C interface. Adfruit created a breakout board utilizing the TCS34725 color sensor.

TCS34725 Breakout from Adafruit
TCS34725 Breakout from Adafruit

In this project, I am using the color sensor on the palm of a glove to sense colors when the wearer moves it over colored objects and makes a grabbing gesture.

Software Components

  1. Arduino Ide 1.5.8
  2. Eclipse Ponte MQTT HTTP Bridge
  3. Adafruit TCS34725 Library
  4. Software Serial Library

The Bread Board Proof of Concept

Almost all the software components were done using Arduino Uno and tested on the bread board. Didn’t test it out with the Pro Trinket. Big mistake as i came to realize two days later. See below at the software section.

TCS34725, ESP8266 and Arduino on breadboard
TCS34725, ESP8266 and Arduino on breadboard

The Perfboard Schematic:

This is the first time I put together something on a perfboard and ergo first time using Fritzing for laying out a perfboard. Bear with me on noob mistakes. But I think schematic can get you on the way to reproducing this if you like to.

LightGlove schematic with ESP8266, Pro Trinket , TCS34725
LightGlove schematic with ESP8266, Pro Trinket , TCS34725
The Build

Apart from wanting to make everything work, I also wanted to make sure my valuable Pro Trinket, ESP8266, TCS34725 and other components are removable. My skills at desoldering are next to none and didn’t want to take the risk of losing anything. Initially i was planning to use conductive thread and sew the components directly on the glove. But after testing one part i realized that the conductive thread has too much resistance resulting in halving the input voltage. So i had to rip it all off and then take the plunge on designing a proto board and solder everything. I am glad i did as I learnt so many things which i would never have otherwise.

I am a noob on electrical design. So I do not trust myself yet to create a voltage divider. ESP8266 needs a 3.3v input voltage. The Pro Trinket 5V doesn’t have a 3.3v out. So I bought a NTE 956 voltage regulator but i was running out of time for learning and applying proper voltage dividing techniques. So it was a pleasant surprise when i noticed that the Adafruit color sensor had a 3.3v pin out apart from the Vin. Checked the voltage on it and it was a good 3.3 v pin. Still was not sure whether the ESP8266 would work reliably as so many of the documentation out there was talking about the WiFi cheap being current hungry. I had no other choice. If the power was not enough, then i had plans to take the voltage from the VBus in Pro Trinket (direct from usb battery) and send it thru a voltage divider. Luckily for me ESP8266 worked with just the 3.3v from the the Color sensor.

The sockets for Pro Trinket and ESP8266:

Sockets for Pro Trinket and ESP8266
Sockets for Pro Trinket and ESP8266

 

The wire leads for Adafruit TCS34725 color sensor (5v Version).. On the right the backside of the protoboard with all the connections routed. Getting ready for soldering the wire leads for the LEDs and the Flux sensor.

Leads for TCS34725
Leads for TCS34725
reverse side of protoboard
reverse side of protoboard.

 

 

Connections for LED and Flux Sensor
Connections for LED and Flux Sensor
Reverse after all the soldering
Reverse after all the soldering

 

 

 

Preparing the LED and Flux Sensor mini protoboard that will go on the backside of the glove.

LED and Flux Sensor soldering
LED and Flux Sensor soldering

 

 

 

 

 

 

 

Finally done with all the soldering. Ready for sewing..

 

Soldering Complete
Soldering Complete

 

 

 

 

 

 

After the sewing, the glove looked like this: (Palm/Inside Wrist and Back/finger with Flux sensor). Note that i have inserted the Pro Trinket and the ESP8266 on the respective sockets. The color sensor had male pins which i soldered for bread board testing which i didn’t want to remove. So i just pushed it through the fabric and connected the wire on the inside which made a temporary lock. I also used a jumper to connect the LED Pin to the Interrupt Pin on the Color sensor.

TCS34725 sewn inside the palm
TCS34725 sewn inside the palm
Flux Sensor or LED on the back of the hand
Flux Sensor or LED on the back of the hand

 

 

 

 

 

The Software

I tested everything on the bread board with Arduino Uno. I started moving the code to Pro Trinket. At first everything seemed to work correctly as i kept loading and testing software for each of the components. Once i started loading the MQTT part using the ESPRESTHelper library i created, i started noticing weird issues. For using the AT commands with ESP,  the code needs to send the size of the TCP packet first before sending the actual packet. My helper library was using Arduino Strings and it was constantly reading a 0 length packet when using string.length().  Also it kept working perfectly when only the MQTT code was on the board and nothing else. I was worried that something is wrong with the connections/voltage/current etc and i had no way of measuring any of this. I have a multimeter but don’t have the confidence to measure current using it.

Then i started noticing the dynamic memory bytes during the compile time. The pro trinket is supposed to have the same 2Kb dynamic memory as that of Uno. But for some reason, i began to notice that whenever the size exceeded 1024 bytes, weird things happened with strings.

So I had to do 2 things: a) Remove String object from the code. b) Reduce dynamic memory size by using PROGMEM where possible. So I ended up removing my own ESPRESTHelper library which was using String heavily and writing the ESP communication code directly in the sketch using Program. All these reduced the dynamic memory usage to under 1 Kb (When i get some time i will rewrite my ESP REST helper library to remove all references to String.)

Also I didn’t have to include the ESP code to create a WiFi Connection. My ESP already has the fixed IP and automatically connects to my local network when ever its powered on. ( My library has the code for WiFI connection if you need it.)

I wanted to simulate the capture and throw effect. I was able to do this by using the flex sensors bent degrees. If its bent more than 80 degrees, i initiated the color capture. The LED will turn on with the right color. Throwing motion always results in the fingers being stretched fully. That would result in less 15 degrees. So i sent the MQTT message with the RGB payload at this time when the fingers are fully stretched.

(Digression: If some one wants to use this glove in some stage show, it will give them time to do a lengthy dialogue as long as they keep their fingers in a grab position. And at the right moment they can do a throwing motion resulting in the message being sent out and other subscribers to the message doing whatever effects they need to do. I can think of so many Star wars Jedi effects with this.)

Here is the sketch: (note that gamm256.h is a gamma table to drive the gamma correction for the LED. The table is from Adafruit which has an excellent explanation on gamma correction. The file is available at my LightGlove Github repository)

/*******************************************************************************
 * Sketch for Light Glove - A Color catching and throwing glove 
 * Built by: Mohan Palanisamy
 * For: Element14.com 2014 Holiday Lights Road Test
 * Uses parts of code from Adafruit 
 *******************************************************************************/

#include <Wire.h>
#include <Adafruit_TCS34725.h>  //Adafruit_TCS34735.h caused compile error. so removed the underscore.
#include <SoftwareSerial.h>
#include <avr/pgmspace.h>
#include "gamma256.h"

#define FLEXPIN A0
#define ESPRXPIN 4
#define ESPTXPIN 5
#define LEDPINR 9
#define LEDPING 10
#define LEDPINB 11
#define _DEBUG
#define ESP_RX_PIN 4
#define ESP_TX_PIN 5

 extern const uint8_t gamma[];
//Storing in PROGMEM to save dynamic memory space.
//Not the topic name (uri after PUT) and the host name. Change it to your topic and hostname:port.
  PROGMEM  const  char  tcp_packet_template[] ="PUT /resources/FromLightGlove/RGB HTTP/1.1\r\n"
                                       "Host: 192.168.1.217:6000\r\n"
                                      "Accept: */*\r\n"
                                      "Content-Type: application/x-www-form-urlencoded\r\n" ;
 
 //the server name should be surronded by double quote.. so pay attention to the escape charcter for double quote
  const char startTCPSession[] = "AT+CIPSTART=\"TCP\",\"192.168.1.217\",6000"; 

Adafruit_TCS34725 colorSensor = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X);
SoftwareSerial esp8266Serial(ESP_RX_PIN, ESP_TX_PIN);

//Color Capture flags
boolean colorCaptureStarted=false;
boolean colorCaptureFinished=false;
char capturedColorRGB[12];

void setup() {
    Serial.begin(9600);
    colorSensor.begin();
    esp8266Serial.begin(9600);
     esp8266Serial.setTimeout(1000);
     
       pinMode(LEDPINR, OUTPUT);
  pinMode(LEDPING, OUTPUT);
  pinMode(LEDPINB, OUTPUT);
 captureColor();  //do it to turn off the LED and as a sanity check
 //  colorSensor.setInterrupt(true);  // turn off LED
  
}

void loop() {
  
  //Straight value 770
  //Bent 90 degree value 870
  int bentDegrees;
  int rawFlex=analogRead(FLEXPIN);
  bentDegrees=map(rawFlex,770,870,0,90);
  Serial.println(rawFlex);
  Serial.println(bentDegrees);
  
  if(bentDegrees >80 && !colorCaptureStarted)
  {
    Serial.println("Bent..Initiate Color Capture");
    colorCaptureStarted=true;
    captureColor();
    colorCaptureFinished=true;
  }
  
  if(bentDegrees< 15) //throw color if captured
  {
    if(colorCaptureStarted && colorCaptureFinished)
    {
     
      SendMQTTMessage(capturedColorRGB);
      //reset after throwing
      colorCaptureStarted=false;
      colorCaptureFinished=false;
 
    }
  }
  delay(100);
}

void SendMQTTMessage(char* content)
{
    //make sure you adjust this if the template stored in progmem changes depending on your
    //host name, port, topic etc.
   char tcpPacketTemplateBuffer[256];
   strcpy_P(tcpPacketTemplateBuffer,tcp_packet_template);
    
    int contentLength=strlen(content);
 
    sprintf(tcpPacketTemplateBuffer,"%sContent-Length: %d\r\n\r\n%s", tcpPacketTemplateBuffer,contentLength,content);
    
    esp8266Serial.println(startTCPSession);
    
    if (esp8266Serial.find("Linked"))
    {
        esp8266Serial.print("AT+CIPSEND=");
        esp8266Serial.println(strlen(tcpPacketTemplateBuffer));

        esp8266Serial.println(tcpPacketTemplateBuffer);
 
        if (esp8266Serial.find("SEND OK"))
        {
          //After reading close the connection. Do not want to reuse because the TCP stack on
          //ESP8266 might close the connection after a while.
          //So make sure the connection is closed by sending AT+CIPCLOSE
          esp8266Serial.println("AT+CIPCLOSE");
          
          Serial.println("Message Sent.");
        } else
        {
    
          Serial.println("Sending Packet Failed");
        }
  } else
  {
    Serial.println("Cannot Start TCP Session");
   }    
    
   // Serial.println(tcpPacketTemplateBuffer);
   // Serial.println(strlen(tcpPacketTemplateBuffer));
}

void captureColor()
{
  uint16_t clear, red, green, blue;
  memset(capturedColorRGB,0,12);
 
  colorSensor.setInterrupt(false);      // turn on LED
 
  delay(60);  // takes 50ms to read 
  
  colorSensor.getRawData(&red, &green, &blue, &clear);
 
  colorSensor.setInterrupt(true);  // turn off LED
  
  // Figure out some basic hex code for visualization
  uint32_t sum = red+green+blue;
 
  float r, g, b;
  r = red; r /= sum;
  g = green; g /= sum;
  b = blue; b /= sum;
  r *= 256; g *= 256; b *= 256;
   if (r > 255) r = 255;
  if (g > 255) g = 255;
  if (b > 255) b = 255;
  
  sprintf(capturedColorRGB,"%d,%d,%d", (int)r ,(int)g ,(int)b) ;
  setColor( (int)r ,(int)g ,(int)b);
}

void setColor(int red, int green, int blue)
{
  analogWrite(LEDPINR, pgm_read_byte(&gamma[red]));
  analogWrite(LEDPING, pgm_read_byte(&gamma[green]));
  analogWrite(LEDPINB, pgm_read_byte(&gamma[blue]));  
}

This build proved to be the most challenging one i have ever done. From conceptualization to realization, my confidence grew as each obstacle was over come. I am glad and thankful that element14.com choose me as one of the participants in this road test. As we embark on a New Year, I wish every one a happy and prosperous new year. Here’s to us for keep on Making things. Cheers.

Leave a Reply

Your email address will not be published. Required fields are marked *