I did a GET from Firefox to the BMS server that merely echoes back the data you send it, and that worked. But that isn't getting anything processed, so I went to bed last night rather down, and lacking in any ideas on how to make progress.
This morning, I checked my email, and Tom Anderson, of BMS tech support, suggested that I try doing a GET to the test server for processing payments. So I tried that from Firefox, and wow, it worked! For the first time, I got a response that wasn't "Some of the data entered is incorrect. Please retry".
So I thought about this a bit, and it pointed a way forward. I could use lynx (a command line browser in Linux) to send a GET, and then parse the response. I tried it, and I didn't have lynx on DATA1. So I Yummed, and lynx installed, and I tried it using lynx, and it didn't work, except that the BMS server wanted to install a cookie, which I allowed, but no response after that. So I thought a bit, and I thought, well, if not lynx, which is a very old bit of software, maybe something else? I tried elinks, and HEY! It worked! For the first time, I was able to squirt a package of billing data (using the card 4111 1111 1111 1111) to the BMS test server from DATA1 and I got back a response that I could use.
So I was thinking, I can spawn an elinks from my perl program and work with the data I get back, which is a less elegant solution than using LWP (which lets you emulate a browser from within a perl program), but hey, as long as it works, I can live with inelegance. But then I thought, why isn't LWP working when elinks works? So I did an access with the package of data to one of my servers from elinks, and saw what was received (by looking at the server log). And then I did the same thing with LWP, and what was received was ... nothing at all! It saw the access, but not the data package. What the hell? Remember, this all worked fine with the old BMS system.
So I changed the url that I was accessing from
(where $carddata is the package of data, including the card number, expiry date and so on).
Previously, I was putting the package of data into the content of the request. I still don't know why that wasn't working, but this new way? Totally worked!
So, now I can read data from my database, assemble it into the format that BMS Directline wants, send it to their server, and get back a response that I can parse to determine success/failure of the billing; if it's success then the AUTH code (if you don't know the AUTH code then the bank can reverse the billing on the grounds that you didn't get an AUTH code, which happened to me once but I was able to tell them the AUTH code). Not that the AUTH code does anything. It doesn't mean that the billing is authorised, which is what many people think it means. It doesn't mean that the card number exists and has sufficient funds, which is what the banks think it means, because I've had cases where I've gotten AUTH codes when the card didn't actually exist. All it means is that someone along the chain of banks that pass this data along, has assigned an AUTH code. But I digress. If the billing fails then it tells you why, which, in my experince, is one of two reasons. "DECLINED" means you didn't bill the card, and we aren't going to tell you why, and "RETAIN" which means you didn't bill the card, and don't give the card back to the customer. Which for electronic transactions is, of course, nonsense. I think it means "Card is reported stolen".
What this means is that I now have a basis for finishing this project. I still have a fair amount to do; I have to convert three programs from the old style to the new (including changing the parsing of the results), and test them, then get the production server enabled, then try all this on the production server, then, with trembling fnigers and bated breath, move over entirely to the new system.
And hope that BMS don't decide to change it all again next year.