CC3000 - Sending binary data

For other supported Arduino products from Adafruit: Shields, accessories, etc.

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
preusstang
 
Posts: 6
Joined: Wed Apr 23, 2014 6:51 pm

CC3000 - Sending binary data

Post by preusstang »

I was having lockup issues with the cc3000 as a server and decided to try the popular fixes from the library github (https://github.com/adafruit/Adafruit_CC3000_Library):
  • Using SPI_CLOCK_DIV2 in Adafruit_CC3000 constructor,
    #define SEND_NON_BLOCKING 1 in Socket.cpp,
    Modifying the write function as vheun mentioned,
    and adding some delays before closing and after available() calls.
Anyway, these changes have caused some unintended and rather weird side effects. I'm no longer able to send binary data with the "write()" function. For example, this code:

Code: Select all

// Check if request matches a file name
// "start" is the buffer position where the file name starts.
SdFile file;
if ( file.open( _root, start, O_READ ) ) 
{
  Serial.print( start );
  Serial.println( F( " opened!" ) );
  sendFileHeaders( client, start, file.fileSize() );
  Serial.println( F( "Sent file headers." ) );
  delay( 5 );

  int16_t numRead;
  while ( (numRead = file.read( _sdBuffer, SD_BUFFER_SIZE ) ) > 0 )
  {
    client.write( _sdBuffer, numRead );
    delay( 50 );
  }

  file.close();
  Serial.print( start );
  Serial.println( " closed!" );
}
And sendFileHeaders function:

Code: Select all

void sendFileHeaders( Adafruit_CC3000_ClientRef client, char* fileName, uint32_t fileSize )
{
  client.fastrprintln( F( "HTTP/1.1 200 OK"));
  client.fastrprint( F( "Content-Type: " ) );
  if ( endsWith( ".jpg", fileName ) )
    client.fastrprintln( F( "image/jpg" ) );
  else if ( endsWith( ".png", fileName ) )
    client.fastrprintln( F( "image/png" ) );
  else if ( endsWith( ".css", fileName ) )
    client.fastrprintln( F( "text/css" ) );
  else if ( endsWith( ".js", fileName ) )
    client.fastrprintln( F( "text/javascript " ) );
  else if ( endsWith( ".txt", fileName ) )
    client.fastrprintln( F( "text/plain" ) );
  else
    client.fastrprintln( F( "text/html" ) );

  client.fastrprint( F( "Content-Length: " ) );
  client.write( (const void*)&fileSize, 4, 0 );
  client.fastrprintln("");

  client.fastrprintln( F( "Cache-Control: max-age=31449600" ) );
  client.fastrprintln("");
}
Used to work fine for say, an HTML file or a PNG image. fileSize was sent correctly, and text as well as binary image data came out fine in my client browser.
Now, fileSize shows up as squares in firebug/chrome dev console. I now have to convert the file size to a string before sending, like:

Code: Select all

char buffer[16];
sprintf(buffer,"%lu", fileSize);
client.fastrprint( F( "Content-Length: " ) );
client.fastrprintln( buffer );
This allows my HTML files send across just fine as the Content-Length header is sent appropriately. However, my image data is still garbage. Do I really need to covert the entire file to ASCII before sending? That's crazy. I already feel wasteful for having to have a buffer of 16 bytes just to hold a 4 byte value...

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: CC3000 - Sending binary data

Post by adafruit_support_rick »

preusstang wrote:Modifying the write function as vheun mentioned,
Do you have a link to that post?

preusstang
 
Posts: 6
Joined: Wed Apr 23, 2014 6:51 pm

Re: CC3000 - Sending binary data

Post by preusstang »

https://github.com/adafruit/Adafruit_CC ... t-35541053

It really has made the whole project more stable (it hasn't crashed on me yet.) As you can see from my Cache-Control header, I expect very few actual file requests. Most requests are REST type calls to send an IR or RF code and not entirely frequent (using it as a remote control for my TV and lights).

User avatar
tdicola
 
Posts: 1074
Joined: Thu Oct 17, 2013 9:11 pm

Re: CC3000 - Sending binary data

Post by tdicola »

HTTP is a text protocol so int, long, etc. values need to be converted to strings. I'm not sure why you might have been seeing values that worked before--perhaps they were small values where the hex value is the same as the decimal representation. There's no magic in the Arduino libraries either unfortunately, you can see the core Print class code which things like Serial and Client inherit from uses a small buffer to convert numbers into string representations (see the printNumber and printFloat functions towards the bottom). Check out the ltoa, utoa, etc. functions in AVR libc, they're not standard C but they are very useful for converting numeric values to strings. To make things a little easier in your code you might consider adding a sendNumber(long, client&) function that takes a number and client, converts the number to a string (using a small static char buffer internally), and then writes it out to the client with fastrprint.

edit: Sorry should clarify, HTTP headers are a text protocol. For body data like images, etc. that's odd if you're not seeing the data sent. Can you download an image from your server using a tool like wget, curl, etc. and compare the file size, hash, etc. to the image you expect to send? Lets see if something is getting changed or corrupted over the wire.

preusstang
 
Posts: 6
Joined: Wed Apr 23, 2014 6:51 pm

Re: CC3000 - Sending binary data

Post by preusstang »

Thanks for the quick replies. Using wget I receive the following:

Code: Select all

D:\Downloads\wget-1.11.4-1-bin\bin>wget http://192.168.27.101/BANNED.png
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = D:\Downloads\wget-1.11.4-1-bin/etc/wgetrc
--2014-06-22 15:32:43--  http://192.168.27.101/BANNED.png
Connecting to 192.168.27.101:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified
Saving to: `BANNED.png'

    [                                <=>                        ] 9,534       1.33K/s   in 7.6s

2014-06-22 15:32:51 (1.22 KB/s) - `BANNED.png' saved [9534]
So it is not seeing a length. Also, the downloaded size is 9,534KB.
However, the actual file size is 9,533KB and using dev tools in chrome and firefox I see that the correct Content-Length: 9533 header is coming across. I looked at the checksums of both files and they were definitely off.
Actually, I feel stupid now. I just opened up the files in Notepad++ and saw that there's a newline at the beginning of the downloaded one. So, I removed it, saved it, and it opens just fine.
Now I just can't figure out why that extra newline is being added. In my experience, HTTP headers always have an additional newline at the end, so what's wrong with my sendFileHeaders()?

Code: Select all

  void sendFileHeaders( Adafruit_CC3000_ClientRef client, char* fileName, uint32_t fileSize )
  {
    client.fastrprintln( F( "HTTP/1.1 200 OK"));
    client.fastrprint( F( "Content-Type: " ) );
    if ( endsWith( ".jpg", fileName ) )
      client.fastrprintln( F( "image/jpg" ) );
    else if ( endsWith( ".png", fileName ) )
      client.fastrprintln( F( "image/png" ) );
    else if ( endsWith( ".css", fileName ) )
      client.fastrprintln( F( "text/css" ) );
    else if ( endsWith( ".js", fileName ) )
      client.fastrprintln( F( "text/javascript " ) );
    else if ( endsWith( ".txt", fileName ) )
      client.fastrprintln( F( "text/plain" ) );
    else
      client.fastrprintln( F( "text/html" ) );

    char buffer[16];
    sprintf(buffer,"%lu", fileSize);
    client.fastrprint( F( "Content-Length: " ) );
    client.fastrprintln( buffer );

    Serial.print( F( "FileSize=" ) );
    Serial.println( buffer );

    client.fastrprintln( F( "Cache-Control: max-age=31449600" ) );
    client.fastrprintln("");
  }

User avatar
tdicola
 
Posts: 1074
Joined: Thu Oct 17, 2013 9:11 pm

Re: CC3000 - Sending binary data

Post by tdicola »

Ah, good to hear it was a simple issue. Hrm, looking at the header function I don't see anything jumping out at as adding an extra newline. What if instead of using the fastrprintln("") at the end you just send an explicit \r\n with fastrprint("\r\n")? Also just to double check no issues with the data, if you print out the first few bytes of the file to serial as your read them, do you see any newlines there?

preusstang
 
Posts: 6
Joined: Wed Apr 23, 2014 6:51 pm

Re: CC3000 - Sending binary data

Post by preusstang »

I added a print for the first full buffer just for sanity and it looked good (no newlines). Turns out it was the empty call to fastrprintln. It injects an extra newline when you give it an empty string. No idea why.

Anyway, thanks for all your help!

EDIT: It appears that you cannot add an "\r\n" to the end of an existing fastrprintln to make it print two newlines. Doing that caused it to print 3. Very strange. The only way to print a newline is to use:

Code: Select all

fastrprint( "\r\n" );
EDIT2: Even stranger.. using the following

Code: Select all

client.fastrprintln( F( "Cache-Control: max-age=31449600" ) );
client.fastrprint( "\r\n" );
Works fine when accessing via Firefox, but in Chrome the headers don't get the extra CRLF, causing all sorts of mayhem. Using the following:

Code: Select all

client.fastrprintln( F( "Cache-Control: max-age=31449600" ) );
client.fastrprint( "\r\n\r\n" );
makes Chrome behave fine, but in Firefox an extra CRLF appears in the page and in any images, making them unrenderable. (is that a word?)

Locked
Please be positive and constructive with your questions and comments.

Return to “Other Arduino products from Adafruit”