I just got a report that www.flowerbugshop.com isn't attaching JPGs to the e-mails sent out to the users from a project I just uploaded. This is a problem that I saw once before but it was related to white-spaces in the file-names. That's not the problem now, or it shouldn't be since the file-names no longer include white-space.
My first theory is that it is the result of the extension being in all uppercase: JPG instead of jpg. Yup, that's the problem. Shoot, it looks like I set the code to only match files with a 'jpg' extension, no other case is supported. Since I am the one who uploads the files I think I will just make sure they are uploaded with lower case extensions. No reason to add another case to the code.
Problem solved!
Sunday, November 3, 2013
Sunday, April 22, 2012
TCP&UDP, Message Boundaries, Brain Limits
I've been trying to dig into and gain a firm understanding of what it means that TCP lacks message boundaries. I I set up two servers: one UDP and one TCP. The UDP one will only read from the socket when data arrives. Whereas the TCP one reads from it constantly.
I've also confirmed that when UDP packets ate fragmented the largest fragment arrives first. I have not yet figured out, in Perl at least, how to tell if a fragmented packet was received, as well as is more data is coming.
If there is no good way to tell in Perl, or any other language if a UDP packet is fragmented(Wireshark figures it out so maybe C?) then you end up in the same situation as TCP. Though reassembling fragment packets could get interesting in UDP without the additional info. How would you know if one price goes with another? UDP doesn't guarantee the in-order delivery. Even if fragments are received highest byte-offset to lowest what do you do if another packet sneaks in there? Fragments are allowed to find there own route through the network.
It doesn't seem to matter whether you use TCP or UDP. Things are gonna get hairy without some rules being laid down for higher layers to use when reassembling the original message.
Tuesday, April 17, 2012
Installing the OLPC School Server
To install the OLPC School Server you'll first need the latest ISO from the OLPC site. As of this writing version 1.7 is the latest. Once you have the ISO you need to be sure to burn the ISO to the disk using a tool that is able to correctly write the ISO. If you don't use special tools you'll end with a CD holding the ISO file, which won't do you much good. Ubuntu has a pretty good walk through on how to burn an ISO on Windows, Mac OS X and Ubuntu. Just replace their Ubuntu ISO with the OLPC School Server ISO.
Once you've got the CD ready to go, you'll need a computer. I'll refer you to the XO Recommend Hardware page to help with computer selection since I suspect these are likely to change frequently. I'm using a Dell Latitude D610.
Once you have a computer you may need to set the CD-ROM drive as the boot primary boot device. This is a process that can vary wildly based on your computer's make, model and age. It may very well be that your computer is set to boot from CD-ROM first by default. To save some head-aches just put the CD in the drive and restart your computer. If it worked the your monitor will eventually come to rest on the OLPC Server boot screen.
If it didn't work you'll end up loading the usual operating system. This will most likely be the case. In this situation you can either try to search for your model of computer on google and see if other people have figured out the right key to push to get into the boot menu, see if you can catch which button it says to press to enter the settings menu or try alternating between F2, Delete and F12 as the computer starts up. If one of these works you'll find yourself in the BIOS.
At which point things get ever more specific to the machine you're on. First, thing to try is to find something that may be the name and version of the BIOS then google for "changing the boot sequence". If that fails read the on-screen instructions for how to navigate the BIOS. Then look for a menu called "boot sequence", "boot order" or anything involving the word "boot". Once you find the menu review the instructions on the screen for how to change the boot order and set the CD-ROM to be the first device to boot. Finally, exit the BIOS settings screen, being sure to save changes when you leave. If you're concerned you may have mistakenly changed something vital to the system exit without saving and repeat the steps to enter the BIOS and set the CD-ROM as the first boot-able device.
At this point, if everything worked as hoped you should be able to put the OLPC School Server CD in the drive and have it boot to the OLPC School server boot screen pictured above. Once at this screen choose "Install a new system from kickstart". Some cryptic text will appear while the installer loads. While installing I found it sat on a blank blue screen for some time(minutes) before going to graphical installer. At which point the graphical installer should be able to guide you through the rest of the installation.
Unless, like me, it keeps telling you there isn't enough space on the hard drive. I believe I found a solution by deleting all of the current partitions on the drive then telling it to install use all of the space on the disk. I will post a more detail explanation once I'm confident this did the trick.
Once you've got the CD ready to go, you'll need a computer. I'll refer you to the XO Recommend Hardware page to help with computer selection since I suspect these are likely to change frequently. I'm using a Dell Latitude D610.
Once you have a computer you may need to set the CD-ROM drive as the boot primary boot device. This is a process that can vary wildly based on your computer's make, model and age. It may very well be that your computer is set to boot from CD-ROM first by default. To save some head-aches just put the CD in the drive and restart your computer. If it worked the your monitor will eventually come to rest on the OLPC Server boot screen.
 |
| OLPC School Server boot screen |
At which point things get ever more specific to the machine you're on. First, thing to try is to find something that may be the name and version of the BIOS then google for "changing the boot sequence". If that fails read the on-screen instructions for how to navigate the BIOS. Then look for a menu called "boot sequence", "boot order" or anything involving the word "boot". Once you find the menu review the instructions on the screen for how to change the boot order and set the CD-ROM to be the first device to boot. Finally, exit the BIOS settings screen, being sure to save changes when you leave. If you're concerned you may have mistakenly changed something vital to the system exit without saving and repeat the steps to enter the BIOS and set the CD-ROM as the first boot-able device.
At this point, if everything worked as hoped you should be able to put the OLPC School Server CD in the drive and have it boot to the OLPC School server boot screen pictured above. Once at this screen choose "Install a new system from kickstart". Some cryptic text will appear while the installer loads. While installing I found it sat on a blank blue screen for some time(minutes) before going to graphical installer. At which point the graphical installer should be able to guide you through the rest of the installation.
Unless, like me, it keeps telling you there isn't enough space on the hard drive. I believe I found a solution by deleting all of the current partitions on the drive then telling it to install use all of the space on the disk. I will post a more detail explanation once I'm confident this did the trick.
Saturday, April 7, 2012
Performing Range Queries on Amazon's DynamoDB
I’ve been working with Amazon’s DynamoDB quite a bit lately. Just this last week I found myself stumped by the matter of how to perform a range query on the DynamoDB. I knew I needed to use the "BETWEEN" operator but I couldn't sort out how to arrange the array's correctly in the attribute value list. I scoured the official DynamoDB docs and the AWS SDK for PHP docs. I couldn't find any examples of how to perform a range query and the documentation for the nested array of options passed into the query left me a bit muddled.
At this point all I had left were my wits so I started banging at the code until it worked. Thankfully, in the end I figured it out.
The "AttributeValueList" is itself an array that holds two associative arrays. These two arrays specify the type, string or number, as the index. The value of each array is the value to look for. Since the "AttributeValueList" is a regular list type array the first array it holds is the start of the range and the second array is the end of the range.
This sample should be fully executable with that latest(1.5.3) AWS SDK for PHP. I also have samples of how to create a table, update items in the table and retrieve a single item from the table. Though these are very nearly the same as AWS's examples. I made them primarily for demonstration. To use the example from start to end you'll want to put your AWS keys in the props.js file. Then run "createDatabase.php", "updateDatabase.php" and "betweenQuery.php" in that order. The between query should return 2 results.
One other important thing to note. As I currently understand it range key operations can only be performed on a single hash key. Say you have the hash keys 1, 2 and 3 and a range key based on the UNIX epoch recording when an event happened. You'll have to do a range query on each key and merge them on the application side to get a complete picture of all the updates to those keys during a give time range.
With this in mind a friend of mine suggested that hash keys are rather like table names while a table in DynamoDB is more like database. It being understood that like all NoSQL databases you can't do joins across the tables this comparison holds up rather well. This inability to query across multiple hash keys fits with the description of Dynamo in the original paper. The paper outlined how data is partitioned across servers based on an MD5 hash of the hash key. Therefore, if range queries could be performed across all available hash keys considerable load would be levied against the whole cluster. Amazon, understanding this, appears to have put a hard restriction on queries to ensure they can reliably respond to requests within milliseconds.
At this point all I had left were my wits so I started banging at the code until it worked. Thankfully, in the end I figured it out.
The "AttributeValueList" is itself an array that holds two associative arrays. These two arrays specify the type, string or number, as the index. The value of each array is the value to look for. Since the "AttributeValueList" is a regular list type array the first array it holds is the start of the range and the second array is the end of the range.
This sample should be fully executable with that latest(1.5.3) AWS SDK for PHP. I also have samples of how to create a table, update items in the table and retrieve a single item from the table. Though these are very nearly the same as AWS's examples. I made them primarily for demonstration. To use the example from start to end you'll want to put your AWS keys in the props.js file. Then run "createDatabase.php", "updateDatabase.php" and "betweenQuery.php" in that order. The between query should return 2 results.
One other important thing to note. As I currently understand it range key operations can only be performed on a single hash key. Say you have the hash keys 1, 2 and 3 and a range key based on the UNIX epoch recording when an event happened. You'll have to do a range query on each key and merge them on the application side to get a complete picture of all the updates to those keys during a give time range.
With this in mind a friend of mine suggested that hash keys are rather like table names while a table in DynamoDB is more like database. It being understood that like all NoSQL databases you can't do joins across the tables this comparison holds up rather well. This inability to query across multiple hash keys fits with the description of Dynamo in the original paper. The paper outlined how data is partitioned across servers based on an MD5 hash of the hash key. Therefore, if range queries could be performed across all available hash keys considerable load would be levied against the whole cluster. Amazon, understanding this, appears to have put a hard restriction on queries to ensure they can reliably respond to requests within milliseconds.
NetBeans 7.1.1 Per Project PHP Include Path
I wrote the tutorial on how to add PHP libraries to the NetBeans include path assuming it would add it to the PHP include path at the same time. When I ran my test script this was not that case. To resolve that problem a custom php.ini has to be used.
It’s not ideal because it makes adding a library a two part ordeal where mistakes can occur between the NetBeans include path and the php.ini. I might have to do some more research but this appears to be the way it is in NetBeans.
To load a custom php.ini with a custom include path, first copy your current php.ini into your project directory. On Ubuntu 11.10 the php.ini for cli is “/etc/php5/cli/” while the one for apache is in “/etc/php5/apache2/”. Next right click on your project and select “Properties” from the context menu. Then select the “Run Configuration” category.
At this point you have two choices, either set the “Working Directory” to your project directory or specify the full path to your php file in the “PHP Options” field. If you go with the former you’ll have to enter “--php-ini path/to/phpini/php.ini” into the “PHP Options” field. If you had a config directory in the root directory of your project you’d set “PHP Options” to “--php-ini config/php.ini”. In the latter case you’ll have to enter “--php-ini /path/to/project/directory/config/php.ini”.
Now to add your library to the PHP include path. Open the projects php.ini file. Search for “include_path”. You should wind up around line 780. At this point you’ll have two choices commented out either UNIX or Windows. You’ll want to uncomment the line appropriate for your OS. Then append the path to your library. In my case the “include_path” was ".:/usr/share/php:/usr/share/pear” then I added “/home/david/workspace/aws-sdk-for-php” with a colon between the previous entry and the new one. The final line looked like so:
include_path = “.:/usr/share/php:/usr/share/pear:/home/david/workspace/aws-sdk-for-php”
Now when the file is run from NetBeans it will automatically load the library.
Pretty spiffy.
It’s not ideal because it makes adding a library a two part ordeal where mistakes can occur between the NetBeans include path and the php.ini. I might have to do some more research but this appears to be the way it is in NetBeans.
To load a custom php.ini with a custom include path, first copy your current php.ini into your project directory. On Ubuntu 11.10 the php.ini for cli is “/etc/php5/cli/” while the one for apache is in “/etc/php5/apache2/”. Next right click on your project and select “Properties” from the context menu. Then select the “Run Configuration” category.
At this point you have two choices, either set the “Working Directory” to your project directory or specify the full path to your php file in the “PHP Options” field. If you go with the former you’ll have to enter “--php-ini path/to/phpini/php.ini” into the “PHP Options” field. If you had a config directory in the root directory of your project you’d set “PHP Options” to “--php-ini config/php.ini”. In the latter case you’ll have to enter “--php-ini /path/to/project/directory/config/php.ini”.
Now to add your library to the PHP include path. Open the projects php.ini file. Search for “include_path”. You should wind up around line 780. At this point you’ll have two choices commented out either UNIX or Windows. You’ll want to uncomment the line appropriate for your OS. Then append the path to your library. In my case the “include_path” was ".:/usr/share/php:/usr/share/pear” then I added “/home/david/workspace/aws-sdk-for-php” with a colon between the previous entry and the new one. The final line looked like so:
include_path = “.:/usr/share/php:/usr/share/pear:/home/david/workspace/aws-sdk-for-php”
Now when the file is run from NetBeans it will automatically load the library.
Pretty spiffy.
NetBeans 7.1.1 adding a PHP library to a Project
One of the features I really appreciate about using an IDE when programming and NetBeans in particular is code-completion. I have a terrible memory for all of the functions in PHP and even worse for all of the additional libraries I use while programming such as the AWS SDK. Thankfully, NetBeans makes it pretty easy to add a new PHP libraries. The next couple paragraphs will walk you through how to it. I created this tutorial using NetBeans IDE 7.1.1 on Ubuntu so your experience may be slightly different depending on your NetBeans version and OS.
First, put the library in a folder either on its own or with other libraries you want to use. I like putting them in their own folders so I can pick and choose which libraries I want to use for a particular project. Next, in NetBeans, right-click on the project you want to add the library to. Select “Properties” from the context menu. In the “Project Properties” menu select the “PHP Include Path” category. Still in the “Project Properties” menu click “Add Folder”. A file browser dialogue will appear, from there navigate to and select the folder containing the library you want. Once the folder is selected press “OK”.
Now you should see the selected folder in the “PHP Include Path”. Finally click “OK”. At this point there may be a short pause while NetBeans scans the library. After which you’ll be able to use auto-completion for all of the classes and functions in the library. Awesome.
First, put the library in a folder either on its own or with other libraries you want to use. I like putting them in their own folders so I can pick and choose which libraries I want to use for a particular project. Next, in NetBeans, right-click on the project you want to add the library to. Select “Properties” from the context menu. In the “Project Properties” menu select the “PHP Include Path” category. Still in the “Project Properties” menu click “Add Folder”. A file browser dialogue will appear, from there navigate to and select the folder containing the library you want. Once the folder is selected press “OK”.
Now you should see the selected folder in the “PHP Include Path”. Finally click “OK”. At this point there may be a short pause while NetBeans scans the library. After which you’ll be able to use auto-completion for all of the classes and functions in the library. Awesome.
Tuesday, April 3, 2012
A Little Progress on the OLPC Wireless Front
Oh I see, what I called the ‘network scope’ is actually the neighborhood view.
I read the WiFi Connectivity FAQ on the OLPC wiki. They recommend setting the channel to 1.
I change the channel of my wireless access point to 1. Then I try pressing ‘OK’ on the “Wireless Key Required” dialogue. I wait a bit for some kind of response from the system but nothing happens.
I would really like to get the wireless up on here so I can connect to my OLPC server. I have seen some talk suggesting the OLPC can be connected to via the mesh networking capabilities of the OLPC. Currently, I’m running it as a VM on my laptop. I suspect doing so prevents the mesh networking capabilities from working. I might have to see if I can scrounge up another machine to run it on.
To make life simpler I disconnected my wireless encryption. I’m hoping this will help me to provide a baseline. I am attempting to connect to my wireless access point again from the OLPC.
I start out by leaving the neighborhood view then returning back to it. When I get back into it I find all of the previous circles representing nearby wireless access points have disappeared. There are now only three circles: Mesh Network 1, Mesh Network 11 and Mesh Network 6. If I remember correctly from reading the OLPC wireless page, the OLPC first attempts to find a nearby mesh network before looking for wireless access points. It’s only able to operate in either mesh or standard wireless so it has to dedicate time to each when performing discovery.
My wireless access point now appears as a full red circle with a purple border. There is no longer a white lock on a black square in the lower right hand corner of the circle. I clicked on the circle and it began blinking. Then after a couple seconds it is about 80% full and the purple is now filled with white and the purple border has been replaced with thin red borders.
I open the browser to verify the internet is working. I’m able to perform a simple google search for the word “test”, most excellent.
I read the WiFi Connectivity FAQ on the OLPC wiki. They recommend setting the channel to 1.
I change the channel of my wireless access point to 1. Then I try pressing ‘OK’ on the “Wireless Key Required” dialogue. I wait a bit for some kind of response from the system but nothing happens.
I would really like to get the wireless up on here so I can connect to my OLPC server. I have seen some talk suggesting the OLPC can be connected to via the mesh networking capabilities of the OLPC. Currently, I’m running it as a VM on my laptop. I suspect doing so prevents the mesh networking capabilities from working. I might have to see if I can scrounge up another machine to run it on.
To make life simpler I disconnected my wireless encryption. I’m hoping this will help me to provide a baseline. I am attempting to connect to my wireless access point again from the OLPC.
I start out by leaving the neighborhood view then returning back to it. When I get back into it I find all of the previous circles representing nearby wireless access points have disappeared. There are now only three circles: Mesh Network 1, Mesh Network 11 and Mesh Network 6. If I remember correctly from reading the OLPC wireless page, the OLPC first attempts to find a nearby mesh network before looking for wireless access points. It’s only able to operate in either mesh or standard wireless so it has to dedicate time to each when performing discovery.
My wireless access point now appears as a full red circle with a purple border. There is no longer a white lock on a black square in the lower right hand corner of the circle. I clicked on the circle and it began blinking. Then after a couple seconds it is about 80% full and the purple is now filled with white and the purple border has been replaced with thin red borders.
I open the browser to verify the internet is working. I’m able to perform a simple google search for the word “test”, most excellent.
Subscribe to:
Comments (Atom)