Install Flirc software on Fedora (and other RPM based distros?)

The Flirc Media Center Companion is a neat little USB device that lets you use any IR remote control with your Raspberry Pi or Android stick. The software you need to set it up is available for Linux but they only maintain the Debian package. There’s an old RPM available but I had problems installing it.

Here’s what I did to create a working (and current) RPM version:

  1. Download the latest DEB file that matches your architecture here: http://apt.flirc.tv/arch. For example, for 64 bit at this time that’s http://apt.flirc.tv/arch/x86_64/binary/flirc_1.3.6-1_amd64.deb.
  2. Convert it to an RPM with alien: sudo alien -r flirc_1.3.6-1_amd64.deb
  3. Remove the unnecessary and conflicting entries: rpmrebuild -p –change-spec-files=”grep -i flirc” flirc-1.3.6-2.x86_64.rpm

That command will create a new RPM that should install cleanly and it all seems to work fine for me. I’m on Fedora 21 64-bit.

Lazy Admin’s Guide To Changing Mongo Oplog Size

Have you read Mongo’s official guide to changing the size of your oplog http://docs.mongodb.org/manual/tutorial/change-oplog-size/ and found it a bit intimidating? Are you resizing it because you already have replication problems anyway? Might as well rebuild your secondary and increase the oplog size in one shot. It’s basically the same as the procedure outlined here: http://docs.mongodb.org/manual/tutorial/resync-replica-set-member/#automatically-sync-a-member.

1. Set the oplog size in /etc/mongodb.conf. Just add this line (the size is in MB):
oplogSize = 102400

2. Stop your server with:
db.shutdownServer()

3. Empty the data directory – the one set by dbPath. You can remove all the files but it’s probably a better idea to move them to a backup directory in case anything goes wrong.

4. Start your server.

That’s it. The server will find the empty data directory, initialize it using the new oplog size, rejoin the replica set and perform a complete initial sync.

Now, you wouldn’t want to do this on a huge production database but if your database isn’t too large, it’ll save you a bit of reading.

More consistent iteration times

We were doing some quick and dirty load testing the other day, using a simple shell script to load messages into a queue in batches. The code looked something like:

while something ;do
echo Sending messages $(date)
send-messages
some-other-stuff
sleep 5
done

It seemed to be working but every few iterations, the time would skip by 6 seconds instead of 5. Obviously, the time it took to send the messages and do some other stuff was adding up. Since we’re in quick and dirty mode anyway, my first instinct was to run the send-messages asynchronously (the other stuff was printing log output and had to run in sequence), so we just added an & to the end of the send-messages line and the number of skips dropped by about a third.

This was an improvement but we were still skipping pretty often and we realized we could do even better.

Rather than using sleep to add a delay, we realized we could use it to act more like a timer. We started the sleep in the background at the top of the loop body and called wait once the body of the loop was done and ready to pause for the remaining time. It was a simple change:

while something ;do
sleep 5 &
slp=$!
echo Sending messages $(date)
send-messages
some-other-stuff
wait $slp
done

This kept each loop iteration really close to the 5 second goal. We might still see some drift over time but it was good enough for our purposes.

WebSphere Jython scripting, add the script directory to the import path

If you’re running a script with wsadmin and it tries to import other modules that live in the same directory, you’ll discover another difference between Python and wsadmin. Python will always look for modules in the path that the script was run from. wsadmin won’t. This is kind of annoying if you have a few local import files. Sure, you can always add a directory to the import path via the command line but who ever remembers to do that (and who remembers the syntax)? Since we already have the script path (from fix5), we might as well add it to the import path.

if mainfile:
    mainpath = os.path.dirname(mainfile)
    if mainpath:
        sys.path[:0] = [ os.path.abspath(mainpath) ]

See fix5 for the computation of mainfile. Better yet, download the whole collection from here.

Running the latest Bing on “not supported” devices

There’s a new version of Bing available here that supports speech input and turn-by-turn spoken directions. It even seems to have traffic avoidance. I’ve barely had a chance to play with it yet but I wanted to pass along a fix for the “Your device has been identified as not supported.” error you’ll get on any but the very latest phones.

Go to the Bing install directory, mine was “Storage CardProgram FilesBing”, and edit the file “Bing.config”. Find the entry “UseAppServerUpdates”. Change the value from “True” to “False”. Save the file.

That should do it. The application runs now and I don’t get the error anymore.

This was tested on an HTC Kaiser running some 6.5 ROM.

WebSphere Jython scripting, sys.argv[0] and __file__

Today’s problem is that wsadmin sets up sys.argv differently from normal Python (or Jython).  In Python, sys.argv[0] is the name of the script you invoked.  What you’d normally think of as the command line arguments start at index 1.  For whatever reason, wsadmin doesn’t pass the script name.  The arguments are passed in starting at sys.argv[0].  This can be a nuisance if you have a script that you want to run both in wsadmin and Python.  It’s also just one more difference to trip over.

I should give credit to this post for reminding me that this needs fixing and also for pointing out that the full command line is available in the environment.  Thanks!

My code for this is a little clumsy.  Basically, we scan the command line we get from IBM_JAVA_COMMAND_LINE for the -f option.  We  don’t want to pick up just any -f that might be passed as an argument to the script, easy to avoid by stripping off anything after a -- argument.  We also don’t want to pick up a -f that might appear among all the environment data that wsadmin prepends to the command line (unlikely as that might be), so we look for the last -f after removing any -- that might be present.  Even so, if the script name contains a space, we’re not going to get the whole name.  Such is life.

topframe = sys._getframe()
up1frame = topframe.f_back
while topframe.f_back:
    topframe = topframe.f_back

cmdline = os.environ.get('IBM_JAVA_COMMAND_LINE')
if up1frame == topframe and cmdline and not topframe.f_globals.get('__file__'):
    beg = 0
    end = len(cmdline)
    # Throw away everything after the '--', if present.
    dashdash = cmdline.find('--', beg, end)
    if dashdash >= 0:
        end = dashdash
    # If we can't find a filename, just use a "-"
    mainfile = '-'
    dashf = cmdline.rfind(' -f ', beg, end)
    if dashf:
        # Grab everything from after the "-f" to the following space.
        beg = dashf + 4
        space = cmdline.find(' ', beg, end)
        if space >= 0:
            end = space
        mainfile = cmdline[beg:end]
    sys.argv[:0] = [ mainfile ]
    topframe.f_globals['__file__'] = mainfile

By the way, I threw in a little bonus.  In wsadmin, __file__ isn’t normally set for the top-level script.  That last line will set it.

As usual, the complete listing for this evolving bundle of fixes is ibmfixes.py.

Update 4/16/2010: I decided this fix should only be applied if imported directly from the top level. This will keep it from potentially breaking existing main programs that don’t know about it.

WebSphere Jython scripting, __name__ == ‘__main__’

Here’s another small fix.  In Python and even in Jython, the __name__ of the topmost script is ‘__main__’.  This is mostly used in the idiom:

if __name__ == "__main__":
    main()

In wsadmin, __name is set to ‘main’. Rather than put the clumsy:

if __name__ == "__main__" or __name__ == "main":
    main()

… in every script, let’s solve this once and for all:

topframe = sys._getframe()
up1frame = topframe.f_back
while topframe.f_back:
    topframe = topframe.f_back
try:
    if up1frame == topframe and topframe.f_locals['__name__'] == 'main':
        topframe.f_locals['__name__'] = '__main__'
except:
    pass

As usual, the current collection is at http://dbrand666.wordpress.com/ibmfixes-py

Update 4/16/2010: I decided this fix should only be applied if imported directly from the top level. This will keep it from potentially breaking existing main programs that don’t know about it.

WebSphere Jython scripting: bool, True and False

I’m always forgetting that Jython 2.1 doesn’t have a bool type.  It’s so easy to forget and use True or False.  It’s a small thing, but so easy to implement we might as well add it to the collection.  Here you go, a bool type and True and False for Jython 2.1:

try:
    True and False
except NameError:
    class bool(type(1)):
        def __init__(self, val = 0):
            if val:
                type(1).__init__(self, 1)
            else:
                type(1).__init__(self, 0)
        def __repr__(self):
            if self:
                return "True"
            else:
                return "False"

        __str__ = __repr__

    __builtin__.bool = bool
    __builtin__.False = bool(0)
    __builtin__.True = bool(1)

WebSphere Jython scripting, importing the Admin objects

As soon as your Jython scripting reaches the point where you start trying to build reusable Jython modules, you’ll hit another odd limitation of the wsadmin environment – you can only access the Admin objects from the top-level module. Try to create a utility module and import it and you’ll find that AdminTask (or AdminControl, or…) is undefined.

Various workarounds have been posted over the past few years:

  1. Passing the Admin objects into the methods that need them.  This is straightforward but clutters the interface.
  2. Registering the Admin objects as __builtin__.  This does make them available everywhere but it does mean adding (ugly) boilerplate to modules that might not even have much to do with WebSphere internals.
  3. Registering the Admin objects as modules, as for example in this blog post.  This is really just a variant of the previous workaround.  I think it’s a bit cleaner, but it still makes demands on the caller.

To my mind, and for the purpose of the ibmfixes module, the problem with all of these is that the caller has to know that you’re going to need these objects (and which ones).  I want a solution that can be implemented in the module where the object is needed.  More than that, I want to be able to hide the fix in a module to be imported by the module that needs it so I don’t even have to know about it anymore.

Fortunately, Jython provides the required functionality:

import sys

# We're going to be referencing the top-level frame in many of the fixes.
topframe = sys._getframe()
while topframe.f_back:
    topframe = topframe.f_back

try:
    for module in 'AdminApp', 'AdminConfig', 'AdminControl', 'AdminTask', 'Help':
        if topframe.f_globals.has_key(module):
            sys.modules[module] = topframe.f_globals[module]
except:
    pass

What this does is copy the Admin objects from the topmost frame into sys.modules. Once that’s done, any module can import the Admin objects they need as if they were proper modules.  The beauty of this approach is you can now use the Admin objects anywhere without adding any additional burden on the scripts that might call you (or in an existing system, might already be calling you).

My collection of fixes for WebSphere Jython scripting part 1 – os.environ, os.system, os.path.expandvars, …

I’ve been doing a lot of Jython scripting lately for IBM WebSphere v6.1. Jython is a wonderful scripting language but the implementation that ships with WebSphere (even WebSphere v7.0) is Jython 2.1 which is almost a decade old. Mostly that means you need to watch those version compatibility notes when browsing the official Python documentation.

One thing I’ve run across that seems unnecessary is that several of the “os” functions fail on newer Windows systems. Try os.environ[‘windir’] for example.  This goes back to the old Jython version that predates Windows 2003 and Vista. IBM chose to document this limitation: in the WebSphere v7.0 InfoCenter rather than fix it.

I decided to take a look and it really isn’t hard to fix. The problem is in a table in javaos.py:

    _osTypeMap = (
        ( "nt", r"(nt)|(Windows NT)|(Windows NT 4.0)|(WindowsNT)|"
                r"(Windows 2000)|(Windows XP)|(Windows CE)" ),
        ( "dos", r"(dos)|(Windows 95)|(Windows 98)|(Windows ME)" ),
        ( "mac", r"(mac)|(MacOS.*)|(Darwin)" ),
        ( "None", r"(None)" ),
        ( "posix", r"(.*)" ), # default - posix seems to vary mast widely
        )

It would have been so easy for me to go and edit the file but clients tend to frown things like patching the official distribution. I wanted something I could just import. Here’s what I ended up with:

import java, sys

try:
    import javaos
    if javaos._osType == 'posix' and 
            java.lang.System.getProperty('os.name').startswith('Windows'):
        sys.registry.setProperty('python.os', 'nt')
        reload(javaos)
except:
    pass

Admittedly, this depends on the details of the current implementation but I consider that reasonable in this case given that it’s a hack that only applies to the current implementation. I chose to catch and ignore all exceptions so this fix won’t end up breaking a script should the implementation change in the future. Presumably when that happens this fix won’t be needed anymore anyway.

I’ve placed this fix along with others I’ll be writing about later in a file called ibmfixes.py. I include it in every script I write, or at least every script that might run into these issues, and I no longer have to give them any thought. Solve the problem and move on.