Use JSON format for database archive files.

[drgrice1]

Instead of using mysqldump use a JSON format for database dump files in the course archive. Each table has its own dump file as before. Each line of a table dump file is a valid JSON array, however, the lines of the file are not separated by commas nor contained in a parent array (so not quite valid JSON for the file as a whole). The first line of each table dump file is an array of fields in the table at the time that the archive is created. The second line is an array of the types of those fields in the same order as on the first line. The remaining lines of the file are arrays data stored in the table also in the same order as the first two rows.

When unarchiving a table dump file, first the table is created according to the schema of the current release.

A prepared statement for a fixed number of rows is created and used to insert batches of rows from the table, and then any extra rows are inserted with a final specially prepared statement. This gives a considerable speed improvement over inserting a single row at a time.

If a table's structure has changed from the time the archive was made, then upgrades are made in place. This is detected by considering the first two lines of the file and comparing those to the schema of the current release. The upgrades are handled per table. Each database record can override the upgrade_data method to perform custom data upgrades for the table. The default method defined in WeBWorK::DB::Record will take care of dropping removed columns or adding new ones, but more complicated data migration (such as moving data from one field to another) will require an override method. Note that the default upgrade_data method is not called unless a change is detected, but an override method will always be called. That is another efficiency improvement as that detection is done once for the entire table.

These new format database dump files will be in the dbdump directory in the course archive tar ball instead of the mysqldump directory. Course archive files that are unarchived and have the old mysqldump directory will use the old mysqldump backwards compatibility methods. Note that those courses will need to be upgraded once the unarchival process is complete as before.

The point of this change is to get away from using mydqldump, to have a format that can be worked with without an actual database instance, and to have a format that is not tied to a specific type of database (such as mysql).