Progress in [DateData].[Occurrence]

The Busted Import

I did NOT fully resolve the import issue from earlier in the week. However, I think I identified some issues that were causing chaos on the process.

The two biggest issues from the import were:

1. Excess Quotations

During the course of exporting, transforming, editing, and (re)saving the same CSV, there were way too many quotation marks (“) in the document. Things that were said or quoted, and exist naturally in quotation marks, then had extra quotation marks added when exported as a string value.

Original:

“That’s what she said,” Michael Scott said.

Becomes:

“ ”That’s what she said,” Michael Scott said.”

Becomes:

“ “”That’s what she said,” Michael Scott said.”“ or some ridiculous mitosis of quotation mark bullshit.

2. Escape Characters

Needing to denote to MySql that an apostrophe (‘) was not the end of a

string, when it was in fact being used AS an apostrophe.

Original:

“Michael Scott’s birthday”

Becomes:

“Michael Scott\’s birthday”

That backslash, indicating that the apostrophe IS an apostrophe, and not some character with a special use, is so important here.

Moving Ahead

Determined not to lose momentum, I needed to keep making progress on the Occurrences table despite the busted import.

As a sort of, penance to the database gods, this was a good moment to practice the INSERT statement and do some entries manually.

After only about 120m of entries, varying between single insert and multiple inserts, the records for January and February have been entered.

While repetitive, this process is not as banal as one would think. It was equal parts copying and reviewing past entries, before running the INSERT query.

A few typos and some preference tweaks were caught and edited. I am happy with today’s progress.

[DateData].[Occurrence] snip: