Fast data import trick

useful information about restoring large databases đź‘Ť

gabriela.IO

A few weeks ago my friend Frank de Jonge told me he managed to improve an import into a MySQL server down from more than 10 hours to 16 minutes. According to him it had to do with several field types (too long fields to really small data), the amount of indexes, and constraints on the tables. We were talking about 1 million records here. He wondered if it was possible to make it even faster.

The basics

Turns out there are many ways of importing data into a database, it all depends where are you getting the data from and where you want to put it. Let me give you a bit more context: you may want to get data from a legacy application that exports into CSV to your database server or even data from different servers.

If you are pulling data from a MySQL table into another…

View original post 717 more words

Advertisements

Woah! Black Holes in the Movie Interstellar

https://arxiv.org/pdf/1502.03808.pdf

Today’s Software is Terrible

JJJ

Are you a software developer? I am, and everyday I’m embarrassed by my profession.

Every single day, I run across some website, app, program or plugin that is egregiously broken; embarrassingly broken; 5000-developers-with-six-figure-salaries-and-free-catered-lunches-and-still-can’t-get-it-right, broken.

Apps on my phone, tablet, computer, tv, and car, crash constantly, sometimes resulting in actual data loss. We shoved television behind a pay-wall in a cube that buffers and loads more than it presents anything. We broke copy & paste, because who would ever want to paste a password anywhere? Form fields do this shit where they want to autocomplete and autocorrect and autofill 3 different suggestions at once. We connected wrist-watches to the internet to draw doodles back and forth that don’t even send half the time. We hid mechanical engines behind electronics so complex there is noticeable lag driving performance cars. We connected entertainment systems to airplane diagnostic systems, so passengers can see how high…

View original post 859 more words