Lambda Functions Part 1

I’ve embarked on the journey to move the p2t backend to AWS Lambda, as intimated in the previous post. In order to leverage AWS Lambda, the leap to Node.js or Java must be made. Thus on this glorious 4th of July weekend I jumped in with both feet.

For a native binary to work, wrapped by either language option, a static build is necessary, or else the use of RPATH with a dynamic build. The latter seemed like the logical choice, since this was more or less the method du jour already. One main distinction, implied by the use of RPATH, was the need to distribute shared libraries with the binaries, loaded at runtime. I decide to locate them in ../lib, relative to bin. Pretty standard stuff.

I then set out to build said shared libraries, and here some consternation set in. A quick repeat of a build that took place literally thousands of times before now threw up undefined symbols in a key shared library, when linking. Huh; how could this be? Well let’s just see what nm has to see about this, shall we?

[ec2-user@ip-xxx-xx-xx-xxx p2t]$ nm lib/libplot2txt.so | grep imread
U _ZN2cv6imreadERKSsi

Yep, that’s undefined all right. No great surprise since the symbol(s) in question are external to this object, and are defined in the library on the same link line :

LDFLAGS += -L./lib -lplot2txt
LDFLAGS += -lopencv_core -lopencv_imgproc -lopencv_highgui

Weird. I list the library needing the external symbols first, thus the linker should know to look for them in subsequent libraries. As before, perhaps nm can shed some light here :

[ec2-user@ip-xxx-xx-xx-xxx p2t]$ nm lib/*.so.* | grep imread
...
000000000001a2d0 T _ZN2cv6imreadERKNS_6StringEi
...
000000000001a2d0 T _ZN2cv6imreadERKNS_6StringEi
0000000000019a00 t _ZN2cvL7imread_ERKNS_6StringEiiPNS_3MatE

The light bulb goes on eventually; the API has changed between versions of the dependancy, indeed the required symbol(s) are available in a prior version, which I quickly install. Swell. Wish I could get that 30 mins of my life back, spent reading up on linkers. The main problem lay in using system headers for the dependancy, and installed libraries for linking. Of course had I used headers from the build, the problem would have elicited itself during compilation instead. Oh well, hereafter I shall pledge to read all software release notes for new revisisions (yeh, right). I digress.

I plough on undeterred, including the linker option in the makefile for the binaries:

-Wl, -rpath=XORIGIN/../lib

I could go to the trouble of escaping the requisite dollar sign in $ORIGIN, but I choose to use chrpath for expediancy sake:

[ec2-user@ip-xxx-xx-xx-xxx p2t]$ chrpath -r '$ORIGIN/../lib' bin/text_lines.x
bin/text_lines.x: RPATH=XORIGIN/../lib
bin/text_lines.x: new RPATH: $ORIGIN/../lib

Now make sure I’m good to go ie., check that at runtime the dynamic loader knows where to look:

[ec2-user@ip-xxx-xx-xx-xxx p2t]$ readelf -d bin/text_lines.x | more

Dynamic section at offset 0x17cc80 contains 33 entries:

Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libplot2txt.so]
0x0000000000000001 (NEEDED) Shared library: [libopencv_core.so.2.4]
0x0000000000000001 (NEEDED) Shared library: [libopencv_imgproc.so.2.4]
0x0000000000000001 (NEEDED) Shared library: [libopencv_highgui.so.2.4]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib]

And last but not least, I test zipping up/moving,un-zipping the deliverable on a comparable system:

bill@bill-ThinkPad-W530:~/Desktop$ unzip deliverable.zip
Archive: deliverable.zip
inflating: bin/text_lines.x
inflating: lib/libopencv_calib3d.so.2.4
inflating: lib/libopencv_stitching.so
inflating: lib/libplot2txt.so
...
bill@bill-ThinkPad-W530:~/Desktop$ bin/text_lines.x
ERROR: usage is 'plot2txt.x '

Best error message I’ve seen all day. Next time: uploading to AWS and test with S3 containers