linux n00b

Regex to create list of values for SQL statement

Posted in Howto, SQL by downforce on March 19, 2012

In my day I’m currently undertaking a lot of data migrations. One thing I have to do frequently is find a list of records that do and/or don’t exist in one or more tables. This usually involves getting a list of ID’s from a log output text files then doing SQL queries to attempt to find them!

Unfortunately our clients machines are generally Windows based. Thankfully most are kind enough to allow us to use/install Notepad++ (IMOย ย the best notepad software available on Windows).

As such I utilise Regex a lot to find/strip/edit the contents to get the ID’s out and today I had a brainwave I thought I’d share on how to get it out quickly and turn it into a list that you can cut and paste into a WHERE clause of a SQL statement.

In our case the format of the logs are:

<ID><Tab character><Error message in text>

What I want is:

'<ID>',

And I get there with:

s/^(.*)/'\1',/

Or in Notepad++:

Enjoy!

Advertisements
Tagged with:

chmod folders & files individually

Posted in Howto, Linux by downforce on February 15, 2012
Tagged with:

Migrating Magento to new (sub)domain

Posted in Applications, Howto by downforce on August 12, 2010

UPDATE: Well it turns out this fixes the store, but not the downloader/PEAR updater. More work to figure that one out!

I’ve been trialing Magento on and off for a few years and one day I’ll get my butt into gear to migrate my whole store from Zen Cart (which desperately needs to be patched anyway) to Magento, but I’ve been trying to tie it into a new automated pricing script I’ve been developing (also for a year or so).

I’ve also rationalised my count of subdomains on my domain and moved my old magento development environment from dev.domain.com to store.domain.com. Previously migrating domains has been a major PITA for Magento but today I decided to screw around in the database and found it to be rather painless!

The database is in the same location with same username/password so all I needed to do was to update the base_url and here’s all I did:

So a few steps, confirm that the two config_data fields are where you want them:

select * from mag_core_config_data where path like 'web/%/base_url';

I have the following output:

+-----------+---------+----------+-----------------------+---------------------------------+
| config_id | scope   | scope_id | path                  | value                           |
+-----------+---------+----------+-----------------------+---------------------------------+
|        19 | default |        0 | web/unsecure/base_url | http://dev.domain.com/ |
|        20 | default |        0 | web/secure/base_url   | http://dev.domain.com/ |
+-----------+---------+----------+-----------------------+---------------------------------+

So it was a simple case of updating these values:

update mag_core_config_data set value = 'http://store.domain.com/' where path like 'web/%/base_url';

BUT unfortunately it still wasn’t rendering properly. I discovered that the pages get cached! OH NOES!

Oh well, let’s just purge the cache and start again:

#rm -Rf /path/to/magento/install/var/cache

And good to go! Now to upgrade to the latest version.

vim Search and Replace

Posted in Howto by downforce on April 15, 2010

Had some grief tonight with a search and replace string.

I am backing up some videos to an external HDD and some are files and some are folders, so I’ve created a rsync script which uses a include.txt file.

rsync script is as follows:
rsync -avr --progress --files-from=include.txt / /media/Ext\ Storage

Then in my include.txt I have the following:
data/videos/dir1/
data/videos/dir2/
data/videos/file1.avi
data/videos/file2.mkv

Which I obtained from doing a find dump into a file find /data/videos/ > include.txt

Except for the includes.txt, it the directories need to have the trailing slash.

I first off tried to create a regex search and replace in vim which found all rows that didn’t have a file extension. I managed to create a expression which DID find them but couldn’t work out how to then wrap that in a NOT.

So I thought screw it, I’ll add a trailing slash / to every row then strip them out and after 45min of screwing around I finally got it.

:1,$ s/\(\.[a-zA-Z]*\)\/$/\1/c

I used /c just to make sure it wasn’t going to go crazy, and it didn’t!

It’s probably not the most elegant or efficient but I have a sense of achievement all the same ๐Ÿ™‚

Tagged with: , , , ,

Arch Linux and Samsung Unified Linux Driver

Posted in Arch Linux, Howto by downforce on October 22, 2009

I’ve finally taken the plunge back into linux on my main desktop, after a loong looong break.

Everything has been going swimmingly well (and I’ll comment more on this another time) but one of my last ‘issues’ was getting my Samsung SCX-4521F printer working.

The AUR doesn’t explicitly have it, the closest being this.

So I tried using the Samsung driver out of the box and surprise surprise, no support for Arch Linux.

So I got out trusty old vim and added support for Arch Linux. The main issue is that:
1. It doesn’t detect Arch as a distro; and
2. It hardcodes the use of /etc/init.d.

So I just added an extra check for Arch Linux and set that variable then added a IF statement at the end to set a new variable INIT which sets it to rc.d or init.d and then updated all instances of init.d with $INIT. There is still a couple of WARNINGS produced but the driver works ๐Ÿ˜€

THe last thing I needed to change comes care of this article and thats:

10. Open /etc/cups/printers.conf Under “DefaultPrinter scx4x21” change the “DeviceURI” so it looks like this:
DeviceURI file:/dev/usb/lp0

Want to use it?

Download the Samsung Unified Linux Driver from the Samsung site and use this install.sh instead (you’ll need to rename it). The current version of the Samsung ULD is 3.00.37.

Upgrading Bugzilla 2.20 to 3.2.4 with Errno 150 on namedqueries

Posted in Howto by downforce on July 15, 2009

I’ve spent most of this morning trying to figure out how to upgrade our antiquated Bugzilla 2.20 installation to the latest and greatest 3.2.4.

The ./checksetup.pl kept failing with the error message :

Converting table namedqueries… DBD::mysql::db do failed: Error on rename of ‘./bugzilla/#sql-3967_1c’ to ‘./bugzilla/namedqueries’ (errno: 150) [for Statement “ALTER TABLE namedqueries TYPE = InnoDB”] at Bugzilla/DB/Mysql.pm line 337
Bugzilla::DB::Mysql::bz_setup_database(‘Bugzilla::DB::Mysql=HASH(0x9c766d0)’) called at ./checksetup.pl line 144

Running the SQL command SHOW ENGINE INNODB STATUS; within mysql gave me the following bit of useful info:

————————
LATEST FOREIGN KEY ERROR
————————
090715 8:58:13 Error in foreign key constraint of table bugzilla/namedqueries_link_in_footer:
there is no index in referenced table which would contain
the columns as the first columns, or the data types in the
referenced table do not match to the ones in table. Constraint:
,
CONSTRAINT fk_namedqueries_link_in_footer_namedquery_id_namedqueries_id FOREIGN KEY (namedquery_id) REFERENCES namedqueries (id) ON DELETE CASCADE ON UPDATE CASCADE
The index in the foreign key in table is namedqueries_link_in_footer_id_idx
See http://dev.mysql.com/doc/refman/5.0/en/innodb-foreign-key-constraints.html
for correct foreign key definition.

So after a couple hours of research I’ve discovered the problem is that the table namedqueries doesn’t have the column id.

So in mysql:

use bugzilla;
alter table namedqueries add id mediumint(9) NOT NULL AUTO_INCREMENT UNIQUE FIRST;

Re-ran ./checksetup.pl and all was good ๐Ÿ˜€

Now for the next upgrade error!

Re-creating a mdadm RAID1 array

Posted in Administration, Howto by downforce on November 1, 2008

I just converted my last Ubuntu machine to ArchLinux.

Everything went fairly smoothly (with the exception of ever-changing order of HDD’s, which is now fixed without the Live CD thanks to UUID’s) but I couldn’t figure out how to re-create my RAID1 mdadm array.
Googling away actually didn’t return what I wanted! I was starting to panic, as I didn’t quite think out my backup routine quite as well as I should have (i.e. don’t backup to a software-raid-array!) I sorta needed to get it restored!

Thankfully I figured it out:
mdadm --assemble /dev/md5 /dev/sdc1 /dev/sdd1

Back in business!

Tagged with: , ,

Finding files from the CLI with regex

Posted in Howto by downforce on May 24, 2007

Found a great article by Carla Schroder called Hone Your Scripting With a Regexp Toolbox.

There are some great hints and tips and I know I’ll refer back to this article many times and use it within a script I’m sort-of working on!

Change your default editor

Posted in Howto by downforce on May 22, 2007

DebainAdmin have come through with another great article on How to change the default editor in Debian Etch.

Once again this works perfectly under Ubuntu too.

Get more Gnome Themes

Posted in Howto, Window Managers by downforce on April 27, 2007

When I ran Windows I was never a theme person. Windows themes just took too much of a hit on system performance. Under Linux, it seems that the impact is far less. As such I decided to look for more GNOME themes, bored with the selection offered by the standard Ubuntu installation.

The best site I’ve found so far is freshmeat.net! Head on over to their Gnome theme sub-category.

There is even a page on how to install them, although I think it’s slightly out-of-date (2002).

To install them in Ubuntu do this:

  • System -> Preferences -> Themes
  • Click on Install
  • Browse to the .tar.gz file and import

Now I hit a little snag. Some of them said Installed correctly and sometimes I got a message to choose between Revert theme or Switch to new theme.

  • When I saw Installed correctly, it appears in theme list.
  • When I saw Switch to new theme, I had to say “Yes” then below the Install button is Save. I then saved the theme so it appeared in the list for later.
  • Now you can have a pretty new desktop. I’ll post back with more Gnome themes sites I find.