Linux – Auto Recovery and Deletion of VIM .swp Files

I finally found the setting that was really limiting the number of connections: net.ipv4.netfilter.ip_conntrack_max. This was set to 11,776 and whatever I set it to is the number of requests I can serve in my test before having to wait tcp_fin_timeout seconds for more connections to become available. The conntrack table is what the kernel uses to track the state of connections so once it's full, the kernel starts dropping packets and printing this in the log:

Jun  2 20:39:14 XXXX-XXX kernel: ip_conntrack: table full, dropping packet.

The next step was getting the kernel to recycle all those connections in the TIME_WAIT state rather than dropping packets. I could get that to happen either by turning on tcp_tw_recycle or increasing ip_conntrack_max to be larger than the number of local ports made available for connections by ip_local_port_range. I guess once the kernel is out of local ports it starts recycling connections. This uses more memory tracking connections but it seems like the better solution than turning on tcp_tw_recycle since the docs imply that that is dangerous.

With this configuration I can run ab all day and never run out of connections:

net.ipv4.netfilter.ip_conntrack_max = 32768
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_orphan_retries = 1
net.ipv4.tcp_fin_timeout = 25
net.ipv4.tcp_max_orphans = 8192
net.ipv4.ip_local_port_range = 32768    61000

The tcp_max_orphans setting didn't have any effect on my tests and I don't know why. I would think it would close the connections in TIME_WAIT state once there were 8192 of them but it doesn't do that for me.

Issue solved, posted here: http://blog.joanmarcriera.es/recover-from-a-2008-non-existing-server-with-legato-networker/

this is a copy paste of the entire post, just in case it goes down.

First of all I must thank IPM(our reseller) for their support and reddit for the help.

Backup softwareused to backup: EMC / Legato Networker

Legato server : windows 2008

Legato clients: same hardware (2 years ago fedora something , now ubuntu )

Ubuntu legato installation following this instructions

Trying to recover from an old client, which is no longer available.

So this is the thing:

On 07/20/2008 we backed up a samba server(fedora something) to a tape , setting 1 year as browse policy and retention policy.

Now this tape is recyclable.

We took down the dns name.

We deleted the legato client configuration.

That legato client was reinstalled and is doing other stuff on ubuntu 10.04, with a different name but same ip. Now, 2 years and some month later somebody ask for the files stored there.

Now we need to recover a folder from 2008 backup, on the fedora-samba-server.

First thing, legato does not show the client name because the config was deleted.

We create it again.

We just set the old dns back on track, pointing the same ip, where the old server was, same MAC address ;) .

We created a new ‘old client configuration’ pointing to the new server. (different legato ip for client “I suppose” )

The ssid where the needed folder is on 2 tapes, 20 and 22.

The index for that backup is on tape 21.

We put this tapes on the jukebox (IBMT4000) — not important for the issue –

All three tapes expired its browsable and recoverable time.

So they are on recyclable. We get the clone id from the ssid with following command:

mminfo -avot -q “ssid=<ssid>” -r cloneid We set the tapes to notrecyclable

nsrmm -S <ssid>/<cloneid> -o notrecyclable We change the retention for the tapes for a future date

nsrmm -S <ssid> -e 01/20/2011 We check the dates are correct :

mminf -avV -q “ssid=<ssid>” -r ssbrowse(26),ssretent(26),savetime

So far its OK.

We close the terminal.

Restart the server, just for being sure.

Finally, we recover the index for that ssid where the folder should be.

nsrck -L7 -t “07/20/2008″ oldservername.domain.org

There, we open the Networker User, select the server, select the old client as source, select the new client as destination.

And this is what I get. imgur image of output

Looks like we will have to recover the clientID from 2008. So we need to solve this as follows. Explanation on how to recover a lost clientID to be able to recover non-existent clients.

find bootstrap of desired time

admmarc@linserv7:~$ mminfo -s aplicweb -a -q “volume=000020L4″ -r “volume,client,ssid,savetime,savesets,level,name”|grep bootstrap 000020L4 aplicweb 1115578763 07/16/2008 60 full bootstrap 000020L4 aplicweb 1098884427 07/17/2008 60 full bootstrap admmarc@linserv7:~$ Stop services

net stop nsrexecd services.msc –> stop networker-power,networker-provisioning,networker-*

Move working directories to whatever you can remember:

My networker is installed on D:\Legato\nsr\

mv mm{,.orig}

mv res{,.orig}

mv index{,.orig}

mv tmp{,.orig} this can be deleted, I bak it up because a half backup is pending.

Start networker:

net start nsrd

net start gstd

Set up library scan for devices when ready –> configure library

load without mount the tape with bootstrap on TApe0

mmrecov

* choose Tape0
* write bootstrap id when requested
* file number ?

-record number ? This will recover stuff on res.R Stop services move folder again res –> res.empty res.R –> res start services

Copy clientID , aliases and identity on a txt file stop legato Put the original folders on their place If you re-created the client, you need to erase it.

Also you need to delete it’s index folder , I have it on D:\legato\nsr\index\teyla.mycompany.org start legato

Now, we create the client backup config using the recovered clientID from the text file.

Then, as we have done before:

nsrck -L7 -t 07/20/2008 teyla.mycompany.org

The client where we recover stuff must have @ on its remote users allowed

Is not necessary to recover to the same OS

I’ve just recovered from a non existing fedora 4 machine to an ubuntu 10.04 , with the following command launched from the destination machine

recover -d /tmp/recover -s legatoserver -S 4034983692 /lv2storage/Projectes/folder

I hope it helps future problems on your side.

thanks.