Hello. I have a new blog.

I've moved this blog to the following URL Kerkness.ca. Thank you for visiting, please update your bookmarks.

Wednesday, April 9, 2008

Creating a Touch Screen Kiosk using Firefox, Ubuntu and Blackbox

Several of the components and posts on this web site have revolved around a business project I've been working on which involves creating a touch screen kiosk for use in a public space. I thought it might be beneficial (at least from my own documenting needs) to provide a post covering the overall process of getting a secure touch screen system up running.

The easiest way to build a kiosk application is to customize a web page to serve as the kiosk interface. Launch the web page in a browser and set the browser to run in full screen mode and voila, instant kiosk. To turn it into a touch screen kiosk all you need to do is buy an LCD touch screen monitor and your all set.

This basic solution might work fine if you're always standing next to the computer and can enable full screen mode every time it reboots and also stop anyone from mucking around with your computer should the browser crash and they get access to the desktop.

Setting up a kiosk which can run in a public space and have reasonable enough security to prevent someone from mucking around with it should the browser crash or computer crash takes a little more thought.

I've addressed the overall solution in two parts.

1) The server/host computer
2) The browser

1) The Server / Host Computer

The computer used to host a touch screen application needs some thoughtful consideration. You want something that will offer good reliable performance and also prevent anyone who might be feeling a little malicious from causing the computer or your application any harm. My touch screen system does NOT have a keyboard attached to it which certainly helps in providing a certain level of security but I still need to take steps to make sure the user cannot get access to the desktop, general file system or any application other than the Firefox browser.

For my solution I decided to run Ubuntu 7.10 and use Blackbox as the default desktop environment. Ubuntu allows me to run an apache http server as well as mysql locally. This keeps the application running very fast, provides me with dynamic data and requires no internet connection. Your kiosk application could be simple straight HTML running locally or running off a remote web server whatever suits your needs. Using Ubuntu (or really any flavor of linux you're comfortable with) keeps your kiosk highly customizable.

Using the Blackbox windows manager instead of the default Gnome or KDE desktop environment allows me to lockout the user from accessing anything other than Firefox and keeps your kiosk as a light weight, fast computer with a single focus.

How to set up the server/host computer

  1. Download and install Ubuntu 7.10
    A default install is fine. The main user you will create during install will be considered a super user. You'll later create a user account specifically for accessing the touch screen application. Click here to get Ubuntu

  2. Install Apache, PHP5 and MySQL (Optional)
    My specific application uses dynamic data and requires both Apache and PHP. Using an Ubuntu computer makes it easy to run these services locally and means my computer doesn't require a dedicated internet connection. Although having one makes remote administration and updates easier. To install these on the computer just reference the UbuntuGuide.org wiki

  3. Create a limited user account
    While your touch screen application is running you'll want to have a limited user account logged into the computer.

    To create this account log into Ubuntu using the account you created during install and select :

    System > Administration > Users and Groups

    From the User settings window select:

    Add User

    From the New User Account window fill in the Basic Settings for your user. For the purpose of this example we will use the username: touchuser. After you've provided a user name and and password select the User Privileges tab. Unselect all options which are not a requirement of your touch screen application. For my application I blocked access to all external hard drives, cdroms, floppies and log monitors.

  4. Install Blackbox Window Manager
    Blackbox is a fast, lightweight and minimal windows manager for the X Window System. You can learn more about it here. Blackbox will be setup as the default desktop for our host computer. I use Blackbox because it helps prevent the touch screen user from having access to any applications unless they are specifically enabled via Blackbox.

    To install blackbox open up the terminal and type the following command.
    sudo apt-get install blackbox blackbox-themes
    Next we need to create a .blackboxrc file and a .blackbox directory. These will be used to define our configuration settings for our desktop and define what applications touchuser has access to. Note: we are going to do this in the home directory of the touchuser account not the account we are logged in as. After we create the file and directory we set permissions on them.
    sudo mkdir /home/touchuser/.blackbox
    sudo touch /home/touchuser/.blackboxrc
    sudo chown touchuser /home/touchuser/.blackbox
    sudo chown touchuser /home/touchuser/.blackboxrc
    sudo chgrp touchuser /home/touchuser/.blackbox
    sudo chgrp touchuser /home/touchuser/.blackboxrc
  5. Define the .blackboxrc File
    Using your favorite text editor add the following definitions to the .blackboxrc file. Note the reference to /home/touchuser/.blackbox/menu. This file will be created in the next step.
  6. session.styleFile: /usr/share/blackbox/styles/Gray
    session.menuFile: /home/touchuser/.blackbox/menu
    session.screen0.workspaces: 1
    session.screen0.workspaceNames: My Touch Screen
    session.fullMaximization: True

    NOTE: If you decide to run Blackbox now you may find yourself unable to complete the rest of the steps in this tutorial. All the steps for setting up the host computer assume you are logged into Ubuntu using the user created during install and running a Gnome session. If you're logged into Blackbox press CTRL + ALT + BACKSPACE to return to the Ubuntu login screen. You can select the 'options' button to change what windows manager is used when you log in.

  7. Create the Blackbox menu file
    By default the Blackbox desktop provides no icons. To launch a program the user needs to right-click on the desktop to see a menu of available applications. We want to provide our own menu settings so that the only application available to our user is Firefox. This is really only of limited use as right-clicking on a touch screen is impossible as far as I know, but in case someone figures out how to do it we don't want them to have access to anything other than the browser.

    Enter the following commands into the terminal to create the menu file and set appropriate permissions.
    sudo touch /home/touchuser/.blackbox/menu
    sudo chgrp touchuser /home/touchuser/.blackbox/menu
    sudo chown touchuser /home/touchuser/.blackbox/menu
    To limit our menu to only provide access to Firefox open up the menu file and add the following.
    [begin] (ArtTouch)
    [exec] (firefox) {firefox}
    [end]
    For more details on configuring blackbox and creating menus see the blackbox wiki

  8. Install iDesk
    Blackbox doesn't by default support desktop icons which is okay since we are building a minimal system. Well what if the Firefox browser crashes and the touch screen user is left starting at an empty desktop with no ability to right-click and relaunch the browser. To provide a solution to this problem we are going to install iDesk which adds icon support to minimal window managers such as Blackbox.

    To install iDesk open a terminal and provide the following command
    sudo apt-get install idesk
    Once iDesk is installed we need to create an .ideskrc file for configuration settings and create an .idesktop folder where we can define our icons.
    sudo touch /home/touchuser/.ideskrc
    sudo mkdir /home/touchuser/.idesktop
    sudo chgrp touchuser /home/touchuser/.ideskrc
    sudo chgrp touchuser /home/touchuser/.idesktop
    sudo chown touchuser /home/touchuser/.ideskrc
    sudo chown touchuser /home/touchuser/.idesktop
    Next open up the .ideskrc file and add the following
    table Config
    Background.Color: #C2CCFF
    end
    table Actions
    Execute[0]: left singleClk
    end
    This is a very minimal use of iDesk configuration options for more see the iDesk Usage Wiki

  9. Create an Icon and define icon commands
    Next we need to create an icon for our desktop. You could use the standard Firefox icon but you're probably best to create a PNG specifically for your touch screen application. Create a PNG file (for example touchicon.png) and stick it in the .idesktop directory. To add the icon to the desktop we need to create a file with a .lnk extension and place it in the .desktop directory also.
    sudo touch /home/touchuser/.idesktop/touchicon.lnk
    sudo chown touchuser /home/touchuser/.idesktop/touchicon.lnk
    sudo chgrp touchuser /home/touchuser/.idesktop/touchicon.lnk
    Open the touchicon.lnk file and add the following. Adjust 'caption', 'tooltip', 'width', 'height' and 'x/y' coordinates to suit your needs.
    table Icon
    Caption: Touch Application
    ToolTip.Caption: Touch Me To Launch
    Command: firefox
    Icon: /home/touchuser/.idesktop/touchicon.png
    Width: 400
    Height: 275
    X: 100
    Y: 100
    end
  10. Create Blackbox startup script
    Now that we have a minimal desktop and we also have an icon for our desktop we need to make sure that iDesk is automatically run when Blackbox runs (by default it doesn't). To do this we are going to create a startup script for Blackbox.

    Create a file called .bbstartup.sh in your touchuser's home directory
    sudo touch /home/touchuser/.bbstartup.sh
    sudo chgrp touchuser /home/touchuser/.bbstartup.sh
    sudo chown touchuser /home/touchuser/.bbstartup.sh
    sudo chmod x+ /home/touchuser/.bbstartup.sh
    Add the following to the .bbstartup.sh file
    #!/bin/sh
    idesk &
    exec blackbox
    Now we need to change the path of exec blackbox in /usr/share/xsessions/blackbox.desktop. Start by making a backup
    sudo cp /usr/share/xsessions/blackbox.desktop /usr/share/xsessions/blackbox.desktop_backup
    Edit /usr/share/xsessions/blackbox.desktop and make the following changes to Exec and TryExec definitions
    [Desktop Entry]
    Encoding=UTF-8
    Name=BlackBox
    Comment=Highly configurable and low resource X11 Window manager
    Exec=/home/touchuser/.bbstartup.sh
    Terminal=False
    TryExec=/home/touchuser/.bbstartup.sh
    Type=Application
  11. Set Auto-login and define Blackbox as default window manager
    The final step in configuring our server/host computer is to set up Ubuntu to automatically log into our touchuser profile and to use Blackbox as the default windows manager. This ensures that if the computer crashes or reboots that it immediately goes into kiosk mode. (note we'll further improve this by making the browser automatically launch later).

    Open the Login Window Preferences by selecting:

    System > Administration > Login Window

    Under the General tab select 'Blackbox' for default session.
    Under the Security tab select 'Enable automatic login' and select the user 'touchuser'
Hopefully after all that you should now have a basic set up to run a kiosk. When the computer boots it should load directly to a Blackbox desktop that allows the user to launch Firefox and only Firefox. In the next step of the process we are going to modify a few settings in Firefox to make sure it loads our touch screen application as it's homepage and automatically launches in full screen mode when the computer boots up.

NOTE: If the computer is running in kiosk mode and you want to be able to login as your super user and get back to a fully functional Gnome desktop all you need to do is plug in a keyboard and press CTRL + ALT + BACKSPACE. This will kill the Blackbox session and bring you to the Ubuntu login screen.

2) The Browser

Now that we have our host computer set up and running we'll want to make a few small changes to the preferences of our Firefox browser so that it performs well as a kiosk client.
  1. Set the Home Page
    This should be pretty obvious. When the browser launches we want it to automatically load our Kiosk application. For my solution I have the Kiosk application running locally as a Flex/PHP application. Your kiosk doesn't have to be Flex, it could be a Flash or simple HTML web page and it could be running remotely or locally. Whatever the case may be you'll want to set the URL location of your application as the homepage for Firefox.

  2. Disabling session restore
    When Firefox crashes or the computer is shutdown before closing the browser, Firefox will by default ask if you want to restore your previous session or start a new one the next time it launches. This is kind of a pointless feature for the kiosk as you'll likely always want it to start a new session. You can disable this feature by logging into your kiosk as 'touchuser' launching Firefox and entering 'about:config' in address bar.

    Find the preference settings for browser.sessionstore.resume_from_crash and browser.sessionstore.resume_session_once and set their values to false.

  3. Getting the browser to automatically launch in fullscreen mode.
    When the browser is launched in order to provide a true kiosk type environment we want it to load in full screen mode giving the user no access to the navigation bar and locked out of most shortcuts. The easiest way to do this is to install one of the many 'Full Screen' extensions available for Firefox.

    The one I recommend is R-Kiosk. For the pure purpose of running a web based Kiosk it does a very good job. While logged in as 'touchuser' Visit this link in Firefox and click the 'Add To Firefox' button.

    NOTE: After this component is added to Firefox you'll be unable to make any preference or configuration changes to Firefox unless you launch it in safe mode. For information on how to run Firefox in safe mode, visit this link.

  4. Setting the browser to automatically launch after boot
    When our kiosk system is booted it will automatically login as with the touchuser profile and load our Blackbox desktop. Now we also want Firefox to automatically launch so that after the system boots the user is presented with our Kiosk application running in all it's full screen glory.

    This is very simple to accomplish by adding one line to the Blackbox startup script we created in step 9 when setting up the host computer. You'll need to log into Ubuntu as the user you defined during install for this step.

    Open /user/touchuser/.bbstartup and add the following line
    #!/bin/sh
    idesk &
    /usr/bin/firefox &
    exec blackbox

If everything has gone as I hope, you should have a fully functioning kiosk system. The only thing left to do is buy yourself a fancy touchscreen LCD monitor and plug it in.

References for this setup can be found at the following links.
HOWTO: A Blackbox Guide
Blackbox Wiki
iDesk Wiki

10 comments:

  1. I enjoyed reading your post. I recently designed a touchscreen kiosk application to include the enclosure design. Since this was my first kiosk, I chose to deploy the kiosk with Windows XP and a kiosk application development program called Kiosk-in-a-box (KBOX). Similar to your kiosk; however, I am using the KBOX program to make browser object calls which pull some of the pages from a remote Ubuntu server running apache2, php, and mysql. While reading your article, I was curious whether you had considered implementing a "watchdog" process whereby the watchdog will detect when/if firefox crashes and would automatically restart the browser without interaction from the user. I implemented a similar process on xp. Also, have you developed a virtual onscreen keyboard to be embedded into the pages you are displaying to the user?

    I am anxious to walk through your howto and build a development box similar to yours and port my app over so that it is 100% linux from kiosk to remote server.

    Check out the pics at http://www.southeasternnetworks.com/gallery/project_gallery.htm

    My virtual onscreen keyboard is modified code By Dmitry Khudorozhkov found at http://www.codeproject.com/KB/scripting/jvk.aspx

    Good Luck with your project!

    Regards,

    Jerry
    jerry.bryan at southeasternnetworks dot com

    ReplyDelete
  2. This is a great guide. I have just started a project for an in-store product video guide and think this will be perfect.
    One question though, what model of touchscreen works with Ubuntu? I can't find any that don't say they need Windows drivers.

    ReplyDelete
  3. @Jerry

    Thank you for the comments. I love your peach kiosk case.

    I haven't set up any type of watchdog program but it is a great idea and I think I'll figure something out. When I do I'll post details.

    For a virtual keyboard I'm using a Flex component which I built myself. My entire Kiosk app is built in Flex so this component is easy to use. See link below.

    http://kerkness.blogspot.com/2008/02/flex-on-screen-qwerty-keyboard.html

    ReplyDelete
  4. @belleyboy

    I've had good luck with UnyTouch monitors. www.unytouch.ca on both windows and linux.

    ReplyDelete
  5. The Kiosk Set up is excellent as I had tried to use. But, I am not able to make the cursor/mount pointer invisible for the touch screen application. I tried to use html style method to hide the cusor but it not work in firefox ( work in I.E only ).

    body style="cursor: url(empty.ani);

    Would you help me that how to make the cursor/mouser pointer invisible ?

    ReplyDelete
  6. @Tony,

    The other thing to look at is the Flex CursorManager. You can create a blank png file and then use the Flex CursorManager to set it as the active Cursor. It works great for me with a Flex based kiosk but only works half the time when compiled as an Adobe Air application. I'm very busy with projects but will try and post examples when I have time. If you google CursorManager you should find some examples.

    ReplyDelete
  7. Spiffy!

    When I want to ensure that firefox is always running, I autostart it with this little bash script:

    #!/bin/bash

    /usr/bin/firefox -P default

    while true ; do
    /usr/bin/firefox -P default
    done

    ReplyDelete
  8. vaughan99@yahoo.comAugust 1, 2008 at 2:48 PM

    I'm very interested in creating a client/server embedded Flex-based application, and this seems like a good model. However, might I suggest using a Projector in full-screen mode instead of a Firefox browser? It would cut down on the setup overhead, has a leaner memory profile, and removes one layer (firefox) that could crash.

    ReplyDelete
  9. @vaughan99, Thank you for the comment. When the option is available I compile my kiosk into an Adobe Air application and run it that way. But I build the interface using flex so this is pretty easy to accomplish. Using a projector is also a good option, but I'm not aware if there are any issues getting dynamic data when deployed in this way.

    ReplyDelete

Thank you for the comments.