I forgot to format my Mac to case-sensitive – this way I solved it

I have a new job, and a new computer, and I have to admit: when I moved to this machine, I was not able to resist the temptation of Catalina, so I installed the newest OSX on the laptop.

Everything was quite nice – well, I could rant for a while about why it was important to drop 32bit compatibility, but eventually solved everything there – but there a major problem emerged: when I checked out a repo of ours, I’ve got the following message:

$ git clone git@gwhatever.it.is:our/repo.git
Cloning into 'repo'...
remote: Counting objects: 20310, done.
remote: Compressing objects: 100% (295/295), done.
remote: Total 20310 (delta 254), reused 536 (delta 209)
Receiving objects: 100% (20310/20310), 280.39 MiB | 2.38 MiB/s, done.
Resolving deltas: 100% (12614/12614), done.
Updating files: 100% (1849/1849), done.
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:

  'src/path/Configfile-1.0.cfg'
  'src/path/configfile-1.0.cfg'

Oh gosh. I have to reformat my disk, and start everything over. Maybe I can create a Time Machine backup, and restore everything? I just don’t feel like it.

Some people tend to have the same mistakes again and again, and it seems I am one of those, but this time, I decided to fight back.

Ok, so what can I do? I can create a new disk image, format it to case sensitive, mount it, and use it as a ‘project’ folder.

$ hdiutil create -size 10g -fs "Case-sensitive Journaled HFS+" -type SPARSE -volname Projects ~/Projects.dmg
$ hdiutil attach ~/Projects.dmg.sparseimage
$ sudo setfile -a V /Volumes/Projects/
$ killall Finder
$ mkdir /Volumes/Projects/projects/
$ ln -s /Volumes/Projects/projects/ ~/projects

So let’s see what is happened here?

  • I created a new disk image with case sensitive filesystem. The type is ‘spare’ which means it will automatically grow when needed, but initially it will take only 50mb on disk.
  • I mounted the image
  • I hide it on the desktop (This volume will be always mounted, so I don’t want to see the icon of it) – for this I set the invisible attibute with setfile, and restarted Finder
  • And finally I created the projects folder inside the volume, and linked it into my home directory.

That’s all folks. For more convenience I added the volume into my login items, so the disk image is automatically mounted when I restart my computer.

$ git clone git@gwhatever.it.is:our/repo.git
Cloning into 'repo'...
remote: Counting objects: 20310, done.
remote: Compressing objects: 100% (295/295), done.
remote: Total 20310 (delta 254), reused 536 (delta 209)
Receiving objects: 100% (20310/20310), 280.39 MiB | 2.38 MiB/s, done.
Resolving deltas: 100% (12614/12614), done.
Updating files: 100% (1849/1849), done.
$

Finally I am able to work under windows – with wsl 2

I have to admit that I was not able to work on windows ever.


I mean I tried, but my workflow was not fit windows. I use the terminal and gnu stuff mostly, I do not use an IDE, I use vim and shell, and that is good for me.


That means I could work with Linux… mostly. But all the time I tried, I hit walls as there were a VPN software I had problems to work with, or some video chat app, or an admin tool for eg. Netware, or simply the computer my employer-provided was not working with Linux, it’s battery drained super fast, or the touchpad did not work well, you name it. And I know that there are machines which has excellent Linux support, like the Dell’s XPS developer edition or some Lenovo machines, but I never had those. – I even had a year back in the early 2000s when I used FreeBSD as my main OS because the PCI modem I had money for wasn’t working good with Linux. (It wasn’t worked well under Windows tbh.)


Back in 2009 I bought my first iMac, and working on that machine was awesome! Everything worked out of the box, there was no hardware which would cause any problem, and when I had to code something, I just opened my terminal and started work in an environment, which was pretty close to a Linux experience. I needed to change a few utils for their GNU replacement, but that’s all. I was able to work on my Mac as I was able to work on my Linux box – but it was way better. Hooray.


Years passed, and my main work environment became OSX, I loved it, I had a fine Macbook Air, and everything worked as it was planned for.


One day I bought a PC for home because I wanted to play video games, and sometimes I tried to work with that, but I was not able to, as my work environment were just not fit there. I tried everything. I tried to run Virtualbox and ssh-d into that, but I hated all the terminal emulators I’ve found. It was simply inconvenient every possible way.


In the meantime, my love with the Mac ended – the ‘new’ series for MacBook pros with touch bar and crappy keyboards just killed the experience for me. I am still working on a Mac because that is still the best option, but the love is gone.


When WSL was released, I tried to use it, but it wasn’t good either. I missed locally starting my daemons such as MySQL or Docker, and it was not fit to my workflow. Of course, I was able to run my services under windows, and access them from my WSL installation, but it wasn’t good. It was a bad experience. The best thing I was able to do to spin up a cloud instance and use my WSL installation to ssh into that.


When I first hear about WSL 2.0 it sounded great to me, so I decided to try it out as early as I can. I rolled into the insider program and I installed a version which had it included and started to work with.


So far it is good. It provides me the Linux experience from the moment I open my terminal. All my workflows and utilities behave the same was as I was used to, and it has a seamless integration to my windows desktop.
`

Dealing with MySQL relay logs when disk space is limited, but the replica is far behind the source

I just encountered the problem described in the title, on a server I don’t have enough free space for create a backup as long the replica is catching up, because the relay logs eats all the available free space as the IO thread works faster than the SQL thread in the replication.

What to do? First, we need to get rid of the current relay logs, and then limit the server to not create too much in the future. The relay logs are not meant for manual deletion so we stop the replica, purge the relay logs, set back the replica to the previous position and then restart the mysqld with the relay_log_space_limit variable set up.

mysql> STOP SLAVE;
mysql> PAGER egrep "Relay_Master_Log_File|Exec_Master_Log_Pos"
mysql> SHOW SLAVE STATUS\G
mysql> RESET SLAVE ALL;
mysql> CHANGE MASTER TO ...
mysql> START SLAVE;
mysql> ^d
bash> echo "relay-log-space-limit = 10737418240" >>/etc/mysql/my.cnf #10G
bash> systemctl mysql restart 

As Jean-François Gagné pointed out (right) this is only applies on a really simple case when there is no GTID configured, no parallel replication exists, the slave is not delayed, no SSL, etc.etc.

What to do with the one-shot scripts

I can’t really answer this question, but I am really curious.

During work, I often have small(ish) tasks to solve, and when it is possible I rather write small one-shot scripts instead of doing some manual labor.

These scripts are rough, they are not made perfect because they don’t have to, they have one purpose, and they have to do it right. Sometimes they have small mistakes in them, sometimes they just aren’t perfect, sometimes they are awful. There are always compromises when creating these like ‘does this task worth that plus hours to put them, or just aim for the solvable problem’.

Let me show this in an example. Today I am working on a script what copies big amounts of data between two servers, I want to do it multithreaded, but I also want to limit the number of threads.

My quick and dirty solution looks like this:
for row in cursor:
while len(threading.enumerate()) > MAXTHREADS + 1:
time.sleep(.1)
worker = Worker(row)

Yes, I know, civilized people don’t use ‘sleep’, because that is a waste of CPU, but hey, I will run this script on an ephemeral EC2 instance, so I really don’t care. This will do it, so deal with it.

But after the migration, I do want to keep this script somewhere because it is possible that I’ll need something like this in the future, and I don’t want to reinvent the wheel then.

My GitHub account would be a perfect place for them, but I am not sure if I can put them there. I mean GitHub account is like a business card (remember Patrick Bateman), I don’t want to put ugly code them – I mean if I want to find a new workplace, then it will be possible that they will check my GitHub stuff, and I don’t want them to think if I am a bad coder.

If I had a paid account (which I don’t have) then I’d put these into a private repo.

Nowadays I put these into gists, but gists are not easy to search, so that is not perfect as well.

I am not sure what would be the best.

 

Going back to my Macbook Air

I started to work at my current company at 2013. In that year I got a maxed-out MacBook Air, which I loved a lot, but it became a bit storm-beaten, I poured some coffee into it (two or three times) – the first one led to a mainboard replacement, but the others were forgotten, and once my son hammered it a bit – few damages on carousel, and a broken charge connector which was repaired.

Last year I got a new 15″ maxed-out Macbook pro, and I really didn’t like it, so I was procrastinating the switchover for a few months after the machine was on my desk – I got the machine around May, and I moved on that around October.

And I bought my old laptop from the company because I liked it.

I use the new machine for three months so I can articulate my opinion about the new MBP series: they’re a glorious piece of shit. The keyboard is awful – the worst one I had for years, my fingertips are hurt a lot – I have to type really strong to get the feedback, and that is painful. The machine is too big for me, it is not convenient to put it into my lap, I have to hold it there, not just put down. The touch bar is useless, but at least it is easy to accidentally touching it, which will lead to funny moments like reloading a webpage which I use, and the apps I am using not getting any advantage from it. For example, I can change my terminals color scheme, or I CAN OPEN A F*CKING MANUAL. That’s just wow. I really needed that, thanks, guys! And as a heavy vim user, I really don’t need a physical escape button for sure.  The force click touchpad is useless as well, but at least I had to learn a new way to interact finder, and I just don’t want to talk about USB-c.

So, I decided to move back on my old MacBook Air.

 

 

PMM and IAM Roles

I started to use Percona Monitoring and Management (PMM) recently because it seems promising, and my friends from Percona always recommending it to try out, and frankly, at first sight, I like it.

There are few things which I am not happy about, but mostly I feel OK – but when it comes to the price/value evaluation it becomes better – it is free.

However I found a really disturbing problem, what is bugging me – it needs AWS credentials to discover hosts on RDS. Let me show you.

Continue reading “PMM and IAM Roles”

OSX and case-sensitive file system

I am really angry now.

A few weeks ago, when I was finished my MySQL backend checker I lost about two hours of work because I wasn’t commit anything to git, but I overwrote the working file with one of my doodle files – which file had the same name but with camel case. I had a default APFS filesystem (on High Sierra) – which is not case sensitive. This was a real amateur mistake I admit it, but the damage was done, I had to recreate everything (actually the second time I was way faster, it took around an hour.)

Continue reading “OSX and case-sensitive file system”

Culture Queries

When doing interviews in the past, I always had hard times to figure out how could I ask about those things which are important to me. Let’s face it when you are in an interview, and you want to know if you are supposed to work in the weekends or not, it is not always to best way to ask about this directly because you’ll have a chance that the company will lie to you about that. So, for example, you can ask instead ‘How responsive are people to emails/Slack over the weekends and after 6pm?’ and then you might have the chance that you’ll have the answer you are looking for.

I know this because I recently read this in this on a page at keyvalues. There are a few really good questions there, so I recommend to check the page – you never know when you’ll need this.