Broken Code Breaking Dreams

The Sterling Pro platform uses a COM object for order placement via its API.  If you’re using C++/CLI to avoid doing all the interop and have an interop DLL built, occasionally you will get a BadImageFormatException reported to you during execution.  Pay no attention to this exception; I walked through and validated that the COM object itself is a 32-bit DLL, and that the binary I was using was a 32-bit DLL.  The problem lies in the fact that if Sterling hasn’t been started, you will get weird exceptions like this that are misreported.

In the actual debugger, you will see an EEMessage exception being reported prior to the BadImageFormatException.  Not being a COM expert, I do not know what the cause of this behavior is.  I do know that you can safely ignore the exception if you just make sure that the base.exe and core Sterling application are already running.

Sometimes you just don’t want to bother interfacing with a 3rd-party COM interface doing all the COM-y stuff that’s required.  The easiest way is using C++/CLI and adding a reference to a type library.  If you happen to have an application that’s written in C++ and you also happen to use boost::regex, if you do a release build and attempt to build your application with /clr turned on, you’re going to end up seeing a problem like this:

1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “void __cdecl boost::re_detail::raise_runtime_error(class std::runtime_error const &)” (?raise_runtime_error@re_detail@boost@@$$FYAXABVruntime_error@std@@@Z)
1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “void __cdecl boost::re_detail::put_mem_block(void *)” (?put_mem_block@re_detail@boost@@$$FYAXPAX@Z)
1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “void * __cdecl boost::re_detail::get_mem_block(void)” (?get_mem_block@re_detail@boost@@$$FYAPAXXZ)
1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “void __cdecl boost::re_detail::verify_options(unsigned int,enum boost::regex_constants::_match_flags)” (?verify_options@re_detail@boost@@$$FYAXIW4_match_flags@regex_constants@2@@Z)
1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl boost::re_detail::w32_transform(unsigned int,char const *,char const *)” (?w32_transform@re_detail@boost@@$$FYA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@IPBD0@Z)
1>VoidRaySTIIQ.obj : error LNK2001: unresolved external symbol “char const * __cdecl boost::re_detail::get_default_error_string(enum boost::regex_constants::error_type)” (?get_default_error_string@re_detail@boost@@$$FYAPBDW4error_type@regex_constants@2@@Z)

Googling for the solution doesn’t yield an obvious solution, but there is one post that led me to the correct course of action, and that link is here:  http://permalink.gmane.org/gmane.comp.lib.boost.devel/226293

The issue is that the calling convention is specified incorrectly when you compile a release build.  For some reason, the library is misconfigured for release operation.  My hack of a solution (probably not optimal, but it worked…) was to just rebuild the library after having modified boost/regex/config.hpp and replacing the part dealing with calling conventions from this:

#if defined(BOOST_MSVC) && (BOOST_MSVC >= 1200) && defined(_MSC_EXTENSIONS)
#if defined(_DEBUG) || defined(__MSVC_RUNTIME_CHECKS) || defined(_MANAGED)
# define BOOST_REGEX_CALL __cdecl
#else
# define BOOST_REGEX_CALL __fastcall
#endif
# define BOOST_REGEX_CCALL __cdecl
#endif

#if defined(__BORLANDC__) && !defined(BOOST_DISABLE_WIN32)
# define BOOST_REGEX_CALL __fastcall
# define BOOST_REGEX_CCALL __stdcall
#endif

#ifndef BOOST_REGEX_CALL
# define BOOST_REGEX_CALL
#endif
#ifndef BOOST_REGEX_CCALL
#define BOOST_REGEX_CCALL
#endif

to this (where I just make everything a __cdecl to avoid hassles with the preprocessor defines):

At some point I will revisit the “correct” solution.   But if you need to get your stuff to work and don’t care about calling convention subtleties, then this should work.  Just be sure to rebuild your boost libraries from scratch after having done this.

I spent a long time trying to figure out this one.  It was a huge waste of time.  I hope this solution helps you.

If you’re using a language other than Python to open your HDF5 file full of tables that you created using PyTables with a compression filter where complib=’bzip2′, forget it (… or at least be prepared to do a LOT of work.)  You’ll see something like this:

HDF5-DIAG: Error detected in HDF5 (1.8.4-patch1) thread 139647180633920:
#000: ../../../src/H5Dio.c line 174 in H5Dread(): can’t read data
major: Dataset
minor: Read failed
#001: ../../../src/H5Dio.c line 404 in H5D_read(): can’t read data
major: Dataset
minor: Read failed
#002: ../../../src/H5Dchunk.c line 1733 in H5D_chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#003: ../../../src/H5Dchunk.c line 2742 in H5D_chunk_lock(): data pipeline read failed
major: Data filters
minor: Filter operation failed
#004: ../../../src/H5Z.c line 996 in H5Z_pipeline(): required filter is not registered
major: Data filters
minor: Read failed
Failed table read.

The reason for this is actually on the PyTables web site, under Optimization Tips (http://pytables.github.com/usersguide/optimization.html):

Be aware that the LZO and bzip2 support in PyTables is not standard on HDF5, so if you are going to use your PyTables files in other contexts different from PyTables you will not be able to read them. Still, see the ptrepack (where the ptrepack utility is described) to find a way to free your files from LZO or bzip2 dependencies, so that you can use these compressors locally with the warranty that you can replace them with Zlib (or even remove compression completely) if you want to use these files with other HDF5 tools or platforms afterwards.

So if you’re like me, got thrilled that you got a bonus on data compression, rushed to use bzip2 compression and are now wondering why you can’t do research (in a language other than Python) on your precious data, now you know.

(I just reverted to zlib and the problems went away, but it forced me to rebuild my datasets.)