Welcome, Guest. Please Login
 
  HomeHelpSearchLogin FAQ Radified Ghost.Classic Ghost.New Bootable CD Blog  
 
Page Index Toggle Pages: 1
Send Topic Print
SSH (Read 7935 times)
LoTGoD
Radmeister
**
Offline


Black eyes...broken fingers

Posts: 53
Ohio


Back to top
SSH
Jun 27th, 2008 at 8:33am
 
Can someone school me on SSH?

I am managing a HUGE website forum, and can't seem  to get phpmyadmin to stay connected long enough to backup a several gigabyte database.

Our webhost has suggested SSH with direct database access.  I was also pointed to Mysqlhotcopy to backup large databases.  In it, they recommend uploading a Perl script to test the installed features of the particular PHP version.

If anyone has a link to a guide on SSH, I would very much appreciate it.  I have some Linux exp., but not enough to be dangerous.  Ihave several offers to have someone do it, but I like to get my hands dirty whenever I can.

Thanks guys!
 
WWW lotgod  
IP Logged
 

Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: SSH
Reply #1 - Jun 27th, 2008 at 2:18pm
 
 
WWW  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: SSH
Reply #2 - Jun 29th, 2008 at 2:24pm
 
WinSCP is an SCP client.  It allows you to transfer files using the ssh protocol.  SSH (Secure SHell) is a secure replacement for telnet.  Putty is a fantastic SSH client for windows.  It simply allows you to connect to the command line on a remote server.

I have a guide that may help you get started with Linux -
http://guides.radified.com/magoo/guides/linux/linux_introduction.html
Rad's link might be better to just find the commands you need, but I provide more background.  My guide is far from perfect and I always appreciate feedback on it.

Let us know if there are any other questions we can answer to help.
 
WWW  
IP Logged
 
LoTGoD
Radmeister
**
Offline


Black eyes...broken fingers

Posts: 53
Ohio


Back to top
Re: SSH
Reply #3 - Jun 30th, 2008 at 10:13am
 
Guys,
Thanks for the help links.  I guess I was a bit off in my request for SSH.  What I really need help in is MySQL backups via SSH.  The site I admin for has a huge DB (over 2GB) and the normal vBulletin backups just time out.

Here is the kicker.  We have a seperate web and DB server.  I have SSH to the web server, and can connect over MySQL to the DB server.  I would like to use mysqlhotcopy to make a copy of the databases, but it can only be run locally.  I also have the choice of mysqldump.  I can run this, but then I have to figure out how to get access to the files on the DB server, since I have no connection info for it.

I have opened a case with our webhost, but they seem less than thrilled to give me access.

I will use your guides, and post my results after I get back in touch with them.

Thanks!
 
WWW lotgod  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: SSH
Reply #4 - Jun 30th, 2008 at 6:16pm
 
I searched Google for 'mysqldump remote server' and hit 'I feel lucky!' and came across something.  This seems to do what you need:
http://www.vbulletin.com/forum/showthread.php?t=93752

Then you can move the backed up file to a safe place via SCP.  Gzipping the file first will save substantial transfer time.  Since a mysqldump file is basically plain text, gzip gives nearly and 8-1 compression ratio, so a 2GB file will end up ~350MB.

 
WWW  
IP Logged
 
LoTGoD
Radmeister
**
Offline


Black eyes...broken fingers

Posts: 53
Ohio


Back to top
Re: SSH
Reply #5 - Jun 30th, 2008 at 7:44pm
 
Magoo,
Thanks for that link.  I got the same answer from our webhost.  I kind of goofed it all up the first time I ran it.  I didn't turn our board off before I ran the backup, and had over 2000 connections hitting the db server at once.  Woops!

I am currently downloading the gzip file of my particular table right now, so I should have everything I need.  Now all I need to do is remove the posts (1.5 million...) that pertain to a specific forum, and I'll be good to go.

Thanks again for the helpful links.
 
WWW lotgod  
IP Logged
 

LoTGoD
Radmeister
**
Offline


Black eyes...broken fingers

Posts: 53
Ohio


Back to top
Re: SSH
Reply #6 - Jul 1st, 2008 at 1:39pm
 
Ended up running some custom SQL in phpmyadmin that moved all the posts I was referencing over to a new table.  Also had to extract the thread information as well.

It was not fun, but I did learn some new SQL commands.
 
WWW lotgod  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: SSH
Reply #7 - Jul 2nd, 2008 at 8:20pm
 
If you don't mind sharing, which SQL commands did you find most useful?  I know SQL basics, but its something I should learn more of...
 
WWW  
IP Logged
 
LoTGoD
Radmeister
**
Offline


Black eyes...broken fingers

Posts: 53
Ohio


Back to top
Re: SSH
Reply #8 - Jul 8th, 2008 at 7:50am
 
Magoo,
No problem.  Here are the SQL commands I ran to get the proper posts moved to the new tables.  I basically had to pull posts that met a specific forum ID (e.g. 60).  These had to be cross referenced from a thread ID table.  All in all, there were over 1.5 million posts that were combined in to one table, and around 60k thread ID's to reference those posts.

On to  the code!!!


The first query:

CREATE TABLE offtopic_post_table
SELECT `post`.* FROM `post`, `thread`
WHERE `post`.`threadid` = `thread`.`threadid` AND `thread`.`forumid` =60


The second query:

CREATE TABLE offtopic_thread_table
SELECT * FROM `thread` WHERE `thread`.`forumid` = 60


As you can see, the first query creates a new table called "offtopic_post_table", and then copies the specific posts to that new table that meet the 3 criteria listed.  In this case, the post.threadid must first be the same as the thread.threadid, and then the thread.froumid must equal 60.

The second query crated the new table "offtopic_thread_table", and then copies all entries to the new table where the thread.forumid = 60.

I would be lying if I said I did this on my own.  I had a DBA helping me through email, and I had to massage his code to work with MySQL, as he had never written a query for that database server type.  It took us just about 2 hours of back and forth emailing to finally get the query  to run correctly.

After the new tables were created and populated, the size of the tables together was around 456MB.  I gzip compressed them before FTP'ing them, and I got them down to ~140MB.  

The next step was to copy them to another site, where the admin there is currently working to incorporate them in to his board as an archive.

Let me know if you need anything else answered.

 
WWW lotgod  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: SSH
Reply #9 - Jul 8th, 2008 at 12:11pm
 
Thanks.  That's interesting.
 
WWW  
IP Logged
 
Page Index Toggle Pages: 1
Send Topic Print