Hi, since you mentioned me, I thought I might chime in.
It does sound quite complex what you are doing.
I have most all of Mike's csv utilities ( www.dlss.ca ) and I have written some simple ones for some data.
We regularly compile and manipulate data via csv files.
However, it's nothing like you are doing with this daily task.
If this were my store, and needed to be done regularly, the questions I would ask myself is - does the data actually need to be updated daily. Would we survive if we did it every other day, or even weekly. (Of course I do not know your system).
If this is something that must be done, and the store is generating enough revenue to warrant the efforts, I would seek some software development to handle the task.
And that's exactly what we have done to accomplish our mission.
Last year alone, I spent over $5,000 in software development and I have been teaching myself some new tricks (lookout - we have a computer coder over here

)
I think with some clever programming you can map those fields and translate that data.
Of course that's about the time your vendor will change their format...
Anyway, I think you get the idea. If it's worthwhile, throw some $$$ at it and take a vacation with all the time you will save.