Raw
posted in Frustration |Well. Who’d'a thunk it? My commentary about my *#@!% raw data being changed had two people wanting to know more!
Here’s the scoop, interesting only to about four of my readers, maybe five: I’m grabbing data from our campus data warehouse via a web report. (This means that other data providers won’t work, sorry, Jane!)
Once upon a time, before there were a series of high-profile data break-ins at colleges and universities (not ours), folks like me on campus were able to just link directly to the accounting system and grab the data as read-only users.
Now, alas, the IT department is more security minded. This is Good! Really! In general. But not for folks like me, small potatoes applications systems analysts working for particular departments, that want to be able to do things with the data.
Because instead of being able to link directly to the data…or even the data warehouse snapshot of the data…now we have to use (ack gasp barf) Business Objects to access the data. And we can’t use the desktop version of BO, we have to use the web tool. I re-iterate: ack gasp barf.
So what once required only an ODBC connection and some of my very own SQL statements now requires:
- Using a generic report available to the entire campus.
- Which can be changed at a moments’ notice.
- Without any warning.
- So I have to open a web browser.
- …run the report…
- …Then save to a local machine as an Excel spreadsheet.
- (Though no doubt I could do the same via code, assuming they didn’t upgrade BO and change the layout and the commands and the name of the report and…)
- Then my code has to open the spreadsheet…
- …Run a query that collects only the data I need…
- …massage that data so the format is correct…
- …and insert that data into a table in my local database so that my users can see data from the accounting system side by side with data from our work order management system for reconciliation purposes. All of which is a major pain, and I wish we didn’t need this reconciliation stuff, but due to a particular decision two years ago, we’re stuck with two systems that we need to ensure are both showing the same numbers.
(FYI: The direct link? I could do 9, 10, and 11, and be done.)
Dudes. This sucks dead toads. Not only did I find out yesterday that they changed the column names at some point in the past, so my code that queries the spreadsheet downloaded from the web doesn’t work anymore (hey, no errors–it just doesn’t insert any data, because the column I was querying on doesn’t exist any more). But today I find out that a transaction detail report that previously showed revenue figures suddenly just ignores any revenue and dumps a zero in instead. Because, hey, we’re a university and nobody gets revenue, right? Har, har, think again.
Dudes. This really sucks dead toads.
And nowhere…nowhere…in all of this was there any kind of warning that the report had been changed. None. Nowhere.
Gah. Gimme back my direct link, dammit. We can see all this stuff just fine using the accounting system’s web interface (ack gack, another web interface, slow and ponderous and irritating as hell), one transaction at a time, so it’s not like we shouldn’t be seeing the info to start with. But Kozmik All forbid anyone should want to actually do something with that information, or see multiple transactions, or, or, or…
Grumble, grumble, grumble. I’m going to be talking to the DW folks to see if we can have them create us a specific-to-our-department report. What a pain. They’ve got months‘ worth of reports to create…any request from us could take months to do. Maybe I’ll try to learn more about BO and create my own report. Even if it took me months, it would be less time.
Grumble.
