In my day I’m currently undertaking a lot of data migrations. One thing I have to do frequently is find a list of records that do and/or don’t exist in one or more tables. This usually involves getting a list of ID’s from a log output text files then doing SQL queries to attempt to find them!
Unfortunately our clients machines are generally Windows based. Thankfully most are kind enough to allow us to use/install Notepad++ (IMO the best notepad software available on Windows).
As such I utilise Regex a lot to find/strip/edit the contents to get the ID’s out and today I had a brainwave I thought I’d share on how to get it out quickly and turn it into a list that you can cut and paste into a WHERE clause of a SQL statement.
In our case the format of the logs are:
<ID><Tab character><Error message in text>
What I want is:
And I get there with:
Or in Notepad++:
UPDATE: Well it turns out this fixes the store, but not the downloader/PEAR updater. More work to figure that one out!
I’ve been trialing Magento on and off for a few years and one day I’ll get my butt into gear to migrate my whole store from Zen Cart (which desperately needs to be patched anyway) to Magento, but I’ve been trying to tie it into a new automated pricing script I’ve been developing (also for a year or so).
I’ve also rationalised my count of subdomains on my domain and moved my old magento development environment from dev.domain.com to store.domain.com. Previously migrating domains has been a major PITA for Magento but today I decided to screw around in the database and found it to be rather painless!
The database is in the same location with same username/password so all I needed to do was to update the base_url and here’s all I did:
So a few steps, confirm that the two config_data fields are where you want them:
select * from mag_core_config_data where path like 'web/%/base_url';
I have the following output:
+-----------+---------+----------+-----------------------+---------------------------------+ | config_id | scope | scope_id | path | value | +-----------+---------+----------+-----------------------+---------------------------------+ | 19 | default | 0 | web/unsecure/base_url | http://dev.domain.com/ | | 20 | default | 0 | web/secure/base_url | http://dev.domain.com/ | +-----------+---------+----------+-----------------------+---------------------------------+
So it was a simple case of updating these values:
update mag_core_config_data set value = 'http://store.domain.com/' where path like 'web/%/base_url';
BUT unfortunately it still wasn’t rendering properly. I discovered that the pages get cached! OH NOES!
Oh well, let’s just purge the cache and start again:
#rm -Rf /path/to/magento/install/var/cache
And good to go! Now to upgrade to the latest version.
Had some grief tonight with a search and replace string.
I am backing up some videos to an external HDD and some are files and some are folders, so I’ve created a rsync script which uses a include.txt file.
rsync script is as follows:
rsync -avr --progress --files-from=include.txt / /media/Ext\ Storage
Then in my
include.txt I have the following:
Which I obtained from doing a find dump into a file
find /data/videos/ > include.txt
Except for the includes.txt, it the directories need to have the trailing slash.
I first off tried to create a regex search and replace in vim which found all rows that didn’t have a file extension. I managed to create a expression which DID find them but couldn’t work out how to then wrap that in a NOT.
So I thought screw it, I’ll add a trailing slash / to every row then strip them out and after 45min of screwing around I finally got it.
I used /c just to make sure it wasn’t going to go crazy, and it didn’t!
It’s probably not the most elegant or efficient but I have a sense of achievement all the same 🙂
I’ve finally taken the plunge back into linux on my main desktop, after a loong looong break.
Everything has been going swimmingly well (and I’ll comment more on this another time) but one of my last ‘issues’ was getting my Samsung SCX-4521F printer working.
The AUR doesn’t explicitly have it, the closest being this.
So I tried using the Samsung driver out of the box and surprise surprise, no support for Arch Linux.
So I got out trusty old vim and added support for Arch Linux. The main issue is that:
1. It doesn’t detect Arch as a distro; and
2. It hardcodes the use of
So I just added an extra check for Arch Linux and set that variable then added a IF statement at the end to set a new variable INIT which sets it to rc.d or init.d and then updated all instances of init.d with $INIT. There is still a couple of WARNINGS produced but the driver works 😀
THe last thing I needed to change comes care of this article and thats:
10. Open /etc/cups/printers.conf Under “DefaultPrinter scx4x21” change the “DeviceURI” so it looks like this:
Want to use it?
Download the Samsung Unified Linux Driver from the Samsung site and use this install.sh instead (you’ll need to rename it). The current version of the Samsung ULD is 3.00.37.
I’ve spent most of this morning trying to figure out how to upgrade our antiquated Bugzilla 2.20 installation to the latest and greatest 3.2.4.
./checksetup.pl kept failing with the error message :
Converting table namedqueries… DBD::mysql::db do failed: Error on rename of ‘./bugzilla/#sql-3967_1c’ to ‘./bugzilla/namedqueries’ (errno: 150) [for Statement “ALTER TABLE namedqueries TYPE = InnoDB”] at Bugzilla/DB/Mysql.pm line 337
Bugzilla::DB::Mysql::bz_setup_database(‘Bugzilla::DB::Mysql=HASH(0x9c766d0)’) called at ./checksetup.pl line 144
Running the SQL command
SHOW ENGINE INNODB STATUS; within mysql gave me the following bit of useful info:
LATEST FOREIGN KEY ERROR
090715 8:58:13 Error in foreign key constraint of table bugzilla/namedqueries_link_in_footer:
there is no index in referenced table which would contain
the columns as the first columns, or the data types in the
referenced table do not match to the ones in table. Constraint:
CONSTRAINT fk_namedqueries_link_in_footer_namedquery_id_namedqueries_id FOREIGN KEY (namedquery_id) REFERENCES namedqueries (id) ON DELETE CASCADE ON UPDATE CASCADE
The index in the foreign key in table is namedqueries_link_in_footer_id_idx
for correct foreign key definition.
So after a couple hours of research I’ve discovered the problem is that the table namedqueries doesn’t have the column id.
So in mysql:
alter table namedqueries add id mediumint(9) NOT NULL AUTO_INCREMENT UNIQUE FIRST;
./checksetup.pl and all was good 😀
Now for the next upgrade error!
I just converted my last Ubuntu machine to ArchLinux.
Everything went fairly smoothly (with the exception of ever-changing order of HDD’s, which is now fixed without the Live CD thanks to UUID’s) but I couldn’t figure out how to re-create my RAID1 mdadm array.
Googling away actually didn’t return what I wanted! I was starting to panic, as I didn’t quite think out my backup routine quite as well as I should have (i.e. don’t backup to a software-raid-array!) I sorta needed to get it restored!
Thankfully I figured it out:
mdadm --assemble /dev/md5 /dev/sdc1 /dev/sdd1
Back in business!
Found a great article by Carla Schroder called Hone Your Scripting With a Regexp Toolbox.
There are some great hints and tips and I know I’ll refer back to this article many times and use it within a script I’m sort-of working on!
DebainAdmin have come through with another great article on How to change the default editor in Debian Etch.
Once again this works perfectly under Ubuntu too.
When I ran Windows I was never a theme person. Windows themes just took too much of a hit on system performance. Under Linux, it seems that the impact is far less. As such I decided to look for more GNOME themes, bored with the selection offered by the standard Ubuntu installation.
The best site I’ve found so far is freshmeat.net! Head on over to their Gnome theme sub-category.
There is even a page on how to install them, although I think it’s slightly out-of-date (2002).
To install them in Ubuntu do this:
- System -> Preferences -> Themes
- Click on Install
- Browse to the .tar.gz file and import
Now I hit a little snag. Some of them said Installed correctly and sometimes I got a message to choose between Revert theme or Switch to new theme.
Now you can have a pretty new desktop. I’ll post back with more Gnome themes sites I find.