How to backup and restore informix db using Onbar tool with ONPSM cloud storage device

backupcloudibminformix

I am having a hard time using the Onbar tool (which relies on Onpsm primary storage manager for Informix db (server 12.1) using the cloud device (AWS s3).

https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.bar.doc/ids_bar_526.htm

  1. On the backup server, I have created a a DBSPOOL and LOGPOOL devices using commands similar to the following
onpsm -D add AMAZON1 \
-g DBSPOOL \
-p HIGHEST \
-t CLOUD \
--url https://ifmx-s3-dev.s3.amazonaws.com \
--provider S3 \
--user AKIAIT1111155555X4PA \ 
--password A2nB21111155555nvTI0X9ZxGzUJNJivoBQY9MrD \
--container ifmx-s3-dev \
--max_part_size 25600
  1. I then setup a cron job to perform a standard, level-0 backup of all online storage spaces and used logical logs.

onbar -b -L 0

onbar -b -l

  1. Reading the following IBM publication which advises to backup the storage catalog tables since are not backed up with the database instance, I set up a cron job to backup the $INFORMIXDIR/etc/psm which is supposed to contain the storage catalog tables

https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.bar.doc/ids_bar_526.htm

Copied $INFORMIXDIR/etc/psm directory and $INFORMIXDIR/etc/ixbar.? from the source server to the target and renamed the ixbar.? file replacing the IBM Informix server name that is used on the source computer with the IBM Informix server name of the target computer.

3- Changed ownership of the copied /psm directory and the ixbar file to user informix

4- Shutdown the db to perform a cold restore

onmode -yuk

5- Checked that the backup objects are there

onpsm -O list 

6- Attempted to restore data and the critical files during the cold restore

onbar -r -cf yes

Got this weird error:

the environment variable BAR_BSALIB_PATH must be set in order to use
the -cf option!

I checked the BAR_BSALIB_PATH in the ONCONFIG file and it is set to
$INFORMIXDIR/lib/libbsapsm.so
Even done this
BAR_BSALIB_PATH="/opt/informix-12.10.fc10/lib/libbsapsm.so"
:/home/informix echo $BAR_BSALIB_PATH
/opt/informix-12.10.fc10/lib/libbsapsm.so

Still the same error!

7- Regenerate the emergency boot file – If the ixbar file is not empty and contains object data, onsmsync -b updates the sysutils database and the ixbar file so that they are in sync.

onsmsync -b

bar_act.log

2018-07-20 00:19:07 25656 25370 (-43207) Unable to open connection to
database server: . 2018-07-20 00:19:07 25656 25370 onsmsync
complete, returning 155 (0x9b) 2018-07-20 00:21:24 25933 25931
/opt/informix-12.10.fc10/bin/onbar_d -r

8- Restore the data

onbar -r

Warning: Parameter's user-configured value was adjusted. (ALARMPROGRAM)
gzip: stdin: unexpected end of file

gzip: stdin: unexpected end of file

9- The server came back in fast recovery mode

10 – bar_dbug.log

GET /awwdst13a/rootdbs/0/rootdbs.377.1 HTTP/1.1

then failure messages like
* Failed writing body (0 != 16347)
* Closing connection 0 2018-07-20 00:21:25 25937 25933 smtranid.c:155 Object Transaction List was empty. 2018-07-20 00:21:27
25933 25931 bar_ixbar_list: enter 2018-07-20 00:21:27 25933 25931
bar_ixbar_list: return 0 (0x00) 2018-07-20 00:21:27 25933 25931
bar_build_timeline: enter 2018-07-20 00:21:27 25933 25931
bar_build_timeline: return 0 (0x00) 2018-07-20 00:21:27 25933 25931
smcatalog.c:521 nsmOpenDatabase: Count: opened = 9, missing = 0.
2018-07-20 00:21:27 25933 25931 smtranid.c:84 Object Transaction List
was empty. 2018-07-20 00:21:27 25941 25933 smcatalog.c:521
nsmOpenDatabase: Count: opened = 9, missing = 0. 2018-07-20 00:21:27
25941 filter write_to_filter: Write failed on parent's output
pipe. errno = 32.. 2018-07-20 00:21:27 25941 filter
write_to_filter: Error writing to handle/fd 17 located at 0x1e05930.

11- bar_act.log

2018-07-20 00:21:27 25945 25933 Successfully connected to Storage
Manager. 2018-07-20 00:21:27 25933 25931 Begin cold level 0 restore
rootdbs (Storage Manager copy ID: 0 377). 2018-07-20 00:21:40 25945
25933 Informix Primary Storage Manager session 867 closed 2018-07-20
00:21:40 25945 25933 The child process for the backup and restore
filter is terminating with exit code 0. 2018-07-20 00:21:41 25933
25931 Completed cold level 0 restore rootdbs.

2018-07-20 00:21:41 25969 25933 Begin cold level 0 restore api7dbs
(Storage Manager copy ID: 0 381). 2018-07-20 00:21:42 25968 25933
Informix Primary Storage Manager session 869 opened. 2018-07-20
00:21:42 25968 25933 Successfully connected to Storage Manager.
2018-07-20 00:21:42 25968 25933 Starting Filter /usr/bin/gunzip.
2018-07-20 00:21:42 25968 25933 Informix Primary Storage Manager
session 869 closed

2018-07-20 00:21:44 25971 25933 Begin cold level 0 restore dat2dbs
(Storage Manager copy ID: 0 380). 2018-07-20 00:21:51 25983 25971
There are no more bytes to read.
ISAM Error = 0.
OS Error = 10 (No child processes). 2018-07-20 00:21:51 25983 25971 XBSA Error: (BSAGetData) A system
error occurred. Aborting XBSA session. 2018-07-20 00:21:51 25983
25971 Informix Primary Storage Manager session 875 closed 2018-07-20
00:21:51 25983 25971 The child process for the backup and restore
filter is terminating with exit code 0. 2018-07-20 00:21:53 25971
25933 Unable to close the storage space restore: The physical restore
was not completed.. 2018-07-20 00:21:53 25971 25933 Process 25971
25933 completed. 2018-07-20 00:21:53 25933 25931 (-43246) The ON-Bar
process 25971 exited with a problem (exit code 131 (0x83),
signal 0).

Best Answer

Finally had the time to make a test with psm and onbar to make an Informix backup to Amazon S3.

The test setup is 2 CentOS 7 virtual machines (machine1 and machine2) with an Informix server on each. The machines can talk to each other and with Amazon S3.
machine1 will have Informix server ifx_a and machine2will have Informix server ifx_b.
Informix 12.10.FC12 was installed on each machine, using the default location (/opt/IBM/informix/12.10/).
It was a root installation, but everything else is run under user informix.

  1. On machine1 I setup and initialize Informix server ifx_a.

    • I have the following environment variables:

      PATH=/opt/IBM/informix/12.10/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
      INFORMIXDIR=/opt/IBM/informix/12.10
      INFORMIXSERVER=ifx_a
      INFORMIXSQLHOSTS=/opt/IBM/informix/12.10/etc/sqlhosts
      ONCONFIG=onconfig.ifx_a
      
    • I have the following parameters in the onconfig.ifx_a file:

      ROOTNAME rootdbs
      ROOTPATH $INFORMIXDIR/tmp/demo_on.rootdbs
      
      SERVERNUM 1
      DBSERVERNAME ifx_a
      
      NETTYPE soctcp,1,50,NET
      
      BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
      BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
      BAR_DEBUG 5
      BAR_MAX_BACKUP 3
      BAR_RETRY 1
      BAR_PROGRESS_FREQ 10
      BAR_BSALIB_PATH $INFORMIXDIR/lib/libbsapsm.so
      BACKUP_FILTER  '/usr/bin/gzip -c'
      RESTORE_FILTER '/usr/bin/gzip -d -c'
      BAR_PERFORMANCE 3
      BAR_CKPTSEC_TIMEOUT 15
      BAR_HISTORY 0
      
      PSM_ACT_LOG      $INFORMIXDIR/tmp/psm_act.log
      PSM_DEBUG_LOG    $INFORMIXDIR/tmp/psm_dbug.log
      PSM_DEBUG        5
      PSM_CATALOG_PATH $INFORMIXDIR/etc/psm
      PSM_DBS_POOL     DBSPOOL
      PSM_LOG_POOL     LOGPOOL
      
      ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh
      
    • The sqlhosts file:

      ifx_a    onsoctcp    machine1.local    15010    k=1
      ifx_b    onsoctcp    machine2.local    15020    k=1
      
    • Create the root dbspace file and set up permissions:

      $ touch /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      $ chown informix:informix /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      $ chmod 660 /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      
    • Initialize the Informix server ifx_a:

      $ oninit -vi
      
      $ onstat -d
      
      IBM Informix Dynamic Server Version 12.10.FC12 -- On-Line -- Up 00:15:50 -- 164468 Kbytes
      
      Dbspaces
      address          number   flags      fchunk   nchunks  pgsize   flags    owner    name
      44c23028         1        0x1        1        1        2048     N  B     informix rootdbs
       1 active, 2047 maximum
      
      Chunks
      address          chunk/dbs     offset     size       free       bpages     flags pathname
      44c23268         1      1      0          150000     85073                 PO-B-- /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
       1 active, 32766 maximum
      
      NOTE: The values in the "size" and "free" columns for DBspace chunks are
            displayed in terms of "pgsize" of the DBspace to which they belong.
      
      
      Expanded chunk capacity mode: always
      
    • Create a new database test_db on the new Informix server:

      $ printf "CREATE DATABASE test_db WITH LOG;" | dbaccess sysmaster
      
  2. Follow Back up to Amazon Simple Storage Service using ON-Bar and the PSM and setup the Amazon S3 bucket and credentials. The following steps are based on the mentioned documentation:

  3. Initialize the psm catalog by just using an onpsm command:

    $ onpsm -C detail
    
    The Informix Primary Storage Manager catalog is not present in the '/opt/IBM/informix/12.10/etc/psm/'
    directory.
    The system will try to create it.
    Informix Primary Storage Manager catalog creation in the '/opt/IBM/informix/12.10/etc/psm/' directory
    succeeded.
    
    Informix Primary Storage Manager State:
    
        PSM Unique ID    : 1532732188
        Catalog Location : /opt/IBM/informix/12.10/etc/psm/
        Catalog State    : Unlocked
        Catalog Owner    : 0
        Catalog Lock Mode: Regular
    
    
    Sessions:
        Session ID       Process ID
    Empty SESSION table.
    
  4. Add the Amazon S3 bucket to the storage pools for dbspaces (DBSPOOL) and logical logs (LOGPOOL):

    • $ onpsm -D add AMAZON1 \
      -g DBSPOOL \
      -p HIGHEST \
      -t CLOUD \
      --url https://ifx-a-cloud-bucket.s3-eu-west-1.amazonaws.com \
      --provider S3 \
      --user UUUUUUUUUUUUUUUUUUUU \
      --password XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
      --container ifx-a-cloud-bucket \
      --max_part_size 25600
      
      $ onpsm -D add AMAZON1 \
      -g LOGPOOL \
      -p HIGHEST \
      -t CLOUD \
      --url https://ifx-a-cloud-bucket.s3-eu-west-1.amazonaws.com \
      --provider S3 \
      --user UUUUUUUUUUUUUUUUUUUU \
      --password XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
      --container ifx-a-cloud-bucket \
      --max_part_size 25600
      
    • $ onpsm -D list
      
      Informix Primary Storage Manager Device List
      
      Type   Prio     Block/Size (MB)  Pool Name     Device Name
      FILE   LOW          --/--        DBSPOOL       /opt/IBM/informix/12.10/backups
      CLOUD  HIGHEST      --/--        DBSPOOL       AMAZON1
      
      FILE   LOW          --/--        LOGPOOL       /opt/IBM/informix/12.10/backups
      CLOUD  HIGHEST      --/--        LOGPOOL       AMAZON1
      
    • With a recently created Amazon S3 bucket, the url should point to the region of the bucket, else Amazon S3 will return a temporary redirect payload and onpsm does not deal nicely with it (it seems to crash).

  5. Make a backup of Informix server ifx_a (in this case I am making a whole system backup, option "-w"):

    - $ onbar -b -w -L 0

    • Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

       2018-07-28 00:45:26.596713 7539  7537 /opt/IBM/informix/12.10/bin/onbar_d -b -L 0 -w
       2018-07-28 00:45:26.757667 7539  7537 Using IBM's PSM version 12.10.FC12 as the Storage Manager. XBSA API version is 1.0.3.
       2018-07-28 00:45:26.933961 7539  7537 Archive started on rootdbs (Requested Level 0).
       2018-07-28 00:45:26.966597 7539  7537 Begin level 0 backup rootdbs.
       2018-07-28 00:45:26.966861 7539  7537 Starting Filter /usr/bin/gzip -c.
       2018-07-28 00:45:26.971701 7542  7539 Successfully connected to Storage Manager.
       2018-07-28 00:45:34.500992 7542  7539 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 00:45:34.507614 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:34.511597 7539  7537 Completed level 0 backup rootdbs (Storage Manager copy ID: 0 1).
       2018-07-28 00:45:34.545924 7539  7537 Archive on rootdbs Completed (Requested Level 0).
       2018-07-28 00:45:34.549264 7539  7537 Begin backup logical log 1.
       2018-07-28 00:45:34.549524 7539  7537 Starting Filter /usr/bin/gzip -c.
       2018-07-28 00:45:34.570435 7561  7539 Successfully connected to Storage Manager.
       2018-07-28 00:45:38.455047 7561  7539 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 00:45:38.461007 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:38.555035 7539  7537 Completed backup logical log 1 (Storage Manager copy ID: 0 2).
       2018-07-28 00:45:38.572049 7539  7537 Begin backup logical log 2.
       2018-07-28 00:45:38.572251 7539  7537 Starting Filter /usr/bin/gzip -c.
       2018-07-28 00:45:38.577040 7576  7539 Successfully connected to Storage Manager.
       2018-07-28 00:45:43.152739 7576  7539 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 00:45:43.159746 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:43.214322 7539  7537 Completed backup logical log 2 (Storage Manager copy ID: 0 3).
       2018-07-28 00:45:43.227226 7539  7537 Begin backup logical log 3.
       2018-07-28 00:45:43.227451 7539  7537 Starting Filter /usr/bin/gzip -c.
       2018-07-28 00:45:43.231582 7592  7539 Successfully connected to Storage Manager.
       2018-07-28 00:45:46.529906 7592  7539 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 00:45:46.536171 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:46.561509 7539  7537 Completed backup logical log 3 (Storage Manager copy ID: 0 4).
       2018-07-28 00:45:46.586104 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:46.659232 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/ixbar.1'.
       2018-07-28 00:45:46.662166 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:46.665362 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/ixbar.1' (Storage Manager copy ID: 0 5).
       2018-07-28 00:45:47.106933 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/oncfg_ifx_a.1'.
       2018-07-28 00:45:47.111936 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:47.117141 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/oncfg_ifx_a.1' (Storage Manager copy ID: 0 6).
       2018-07-28 00:45:47.566010 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/onconfig.ifx_a'.
       2018-07-28 00:45:47.571084 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:47.577958 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/onconfig.ifx_a' (Storage Manager copy ID: 0 7).
       2018-07-28 00:45:48.216676 7539  7537 Begin backup of critical file '/opt/IBM/informix/12.10/etc/sqlhosts'.
       2018-07-28 00:45:48.221868 7539  7537 Successfully connected to Storage Manager.
       2018-07-28 00:45:48.227252 7539  7537 Completed backup of critical file '/opt/IBM/informix/12.10/etc/sqlhosts' (Storage Manager copy ID: 0 8).
       2018-07-28 00:45:48.649965 7539  7537 /opt/IBM/informix/12.10/bin/onbar_d complete, returning 0 (0x00)
      
  6. Setup automatic backup on completion for logical logs.

    • Edit the ALARMPROGRAMscript and change BACKUPLOGS to Y:

      BACKUPLOGS=Y
      
    • Force a logical log completion/change and it will backup all logical logs that need.

      $ onmode -l
      
    • Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

      07/28/18 01:11:02  Logical Log 4 Complete, timestamp: 0x4c867.
      07/28/18 01:11:03  Logical Log 4 - Backup Started
      07/28/18 01:11:05  Logical Log 4 - Backup Completed
      07/28/18 01:11:09  Logical Log 5 Complete, timestamp: 0x55db7.
      07/28/18 01:11:10  Logical Log 5 - Backup Started
      07/28/18 01:11:12  Logical Log 5 - Backup Completed
      07/28/18 01:11:36  Logical Log 6 Complete, timestamp: 0x57a93.
      07/28/18 01:11:37  Logical Log 6 - Backup Started
      07/28/18 01:11:38  Logical Log 6 - Backup Completed
      
  7. Verify that the backup objects are listed in the psm catalog:

    $ onpsm -O list
    
    Informix Primary Storage Manager Object List
    
    Object ID            Date Created         Size (MB) Logical path (name.version)
    1                    2018-07-28 00:45:26        2.5 /ifx_a/rootdbs/0/ifx_a.1
    2                    2018-07-28 00:45:34        1.2 /ifx_a/1/1/ifx_a.1
    3                    2018-07-28 00:45:38        1.4 /ifx_a/1/2/ifx_a.1
    4                    2018-07-28 00:45:43        0.9 /ifx_a/1/3/ifx_a.1
    5                    2018-07-28 00:45:46        0.0 /ifx_a/critical_files/ixbar/ifx_a.1
    6                    2018-07-28 00:45:47        0.0 /ifx_a/critical_files/oncfg/ifx_a.1
    7                    2018-07-28 00:45:47        0.1 /ifx_a/critical_files/onconfig/ifx_a.1
    8                    2018-07-28 00:45:48        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.1
    9                    2018-07-28 01:11:03        0.6 /ifx_a/1/4/ifx_a.1
    10                   2018-07-28 01:11:10        0.6 /ifx_a/1/5/ifx_a.1
    11                   2018-07-28 01:11:37        0.1 /ifx_a/1/6/ifx_a.1
    
  8. Export the psm catalog so it can be moved to machine2.

    • This will create a psm_catalog.exp in the current directory. It will contain text files to be imported into a new catalog.

      $ onpsm -C export
      
    • Another way is just to just copy the psm catalog to machine2.

      • On Informix version 12.10.FC10 the export option is broken for CLOUD type devices, so the catalog must be copied instead.
    • The psm catalog is not part of the backups. It should itself be backup up by an external tool, as per the documentation HCL Informix Primary Storage Manager
  9. On machine2 I setup Informix server ifx_b, but I do not initialize it.

    • I have the following environment variables:

      PATH=/opt/IBM/informix/12.10/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
      INFORMIXDIR=/opt/IBM/informix/12.10
      INFORMIXSERVER=ifx_b
      INFORMIXSQLHOSTS=/opt/IBM/informix/12.10/etc/sqlhosts
      ONCONFIG=onconfig.ifx_b
      
    • I have the following parameters in the onconfig.ifx_b file which are mostly equal to Informix server ifx_a, but I prefer to give it a different SERVERNUM and DBSERVERNAME (it is not required, since they are in separate machines):

      ROOTNAME rootdbs
      ROOTPATH $INFORMIXDIR/tmp/demo_on.rootdbs
      
      SERVERNUM 2
      DBSERVERNAME ifx_b
      
      NETTYPE soctcp,1,50,NET
      
      BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
      BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
      BAR_DEBUG 5
      BAR_MAX_BACKUP 3
      BAR_RETRY 1
      BAR_PROGRESS_FREQ 10
      BAR_BSALIB_PATH $INFORMIXDIR/lib/libbsapsm.so
      BACKUP_FILTER  '/usr/bin/gzip -c'
      RESTORE_FILTER '/usr/bin/gzip -d -c'
      BAR_PERFORMANCE 3
      BAR_CKPTSEC_TIMEOUT 15
      BAR_HISTORY 0
      
      PSM_ACT_LOG      $INFORMIXDIR/tmp/psm_act.log
      PSM_DEBUG_LOG    $INFORMIXDIR/tmp/psm_dbug.log
      PSM_DEBUG        5
      PSM_CATALOG_PATH $INFORMIXDIR/etc/psm
      PSM_DBS_POOL     DBSPOOL
      PSM_LOG_POOL     LOGPOOL
      
      ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh
      
    • The sqlhosts file:

      ifx_a    onsoctcp    machine1.local    15010    k=1
      ifx_b    onsoctcp    machine2.local    15020    k=1
      
    • Create the root dbspace file and set up permissions:

      $ touch /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      $ chown informix:informix /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      $ chmod 660 /opt/IBM/informix/12.10/tmp/demo_on.rootdbs
      
  10. Copy into machine2 the Informix server ifx_a psm catalog export.

  11. Import the Informix server ifx_a psm catalog. The imported catalog will assume all the same absolute paths from Informix server ifx_a. But both Informix servers were setup to have the same absolute paths.

    • In the directory where the psm_catalog.exp is, I run:

      $ onpsm -C import
      
       The Informix Primary Storage Manager catalog is not present in the '/opt/IBM/informix/12.10/etc/psm/'
       directory.
      The system will try to create it.
       Informix Primary Storage Manager catalog creation in the '/opt/IBM/informix/12.10/etc/psm/' directory
       succeeded.
      
      This option replaces the Informix Primary Storage Manager catalog using the
       contents of the
      'psm_catalog.exp' directory.
      Are you sure you want to continue? (y/n) y
      
    • Check the psm catalog:

      $ onpsm -C detail
      
      Informix Primary Storage Manager State:
      
          PSM Unique ID    : 1532739005
          Catalog Location : /opt/IBM/informix/12.10/etc/psm/
          Catalog State    : Unlocked
          Catalog Owner    : 0
          Catalog Lock Mode: Regular
      
      
      Sessions:
          Session ID       Process ID
      Empty SESSION table.
      
      
      $ onpsm -O list
      
      Informix Primary Storage Manager Object List
      
      Object ID            Date Created         Size (MB) Logical path (name.version)
      1                    2018-07-28 00:45:26        2.5 /ifx_a/rootdbs/0/ifx_a.1
      2                    2018-07-28 00:45:34        1.2 /ifx_a/1/1/ifx_a.1
      3                    2018-07-28 00:45:38        1.4 /ifx_a/1/2/ifx_a.1
      4                    2018-07-28 00:45:43        0.9 /ifx_a/1/3/ifx_a.1
      5                    2018-07-28 00:45:46        0.0 /ifx_a/critical_files/ixbar/ifx_a.1
      6                    2018-07-28 00:45:47        0.0 /ifx_a/critical_files/oncfg/ifx_a.1
      7                    2018-07-28 00:45:47        0.1 /ifx_a/critical_files/onconfig/ifx_a.1
      8                    2018-07-28 00:45:48        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.1
      9                    2018-07-28 01:11:03        0.6 /ifx_a/1/4/ifx_a.1
      10                   2018-07-28 01:11:10        0.6 /ifx_a/1/5/ifx_a.1
      11                   2018-07-28 01:11:37        0.1 /ifx_a/1/6/ifx_a.1
      12                   2018-07-28 01:32:40        2.5 /ifx_a/rootdbs/0/ifx_a.2
      13                   2018-07-28 01:32:47        0.5 /ifx_a/1/7/ifx_a.1
      14                   2018-07-28 01:32:49        0.0 /ifx_a/critical_files/ixbar/ifx_a.2
      15                   2018-07-28 01:32:49        0.0 /ifx_a/critical_files/oncfg/ifx_a.2
      16                   2018-07-28 01:32:50        0.1 /ifx_a/critical_files/onconfig/ifx_a.2
      17                   2018-07-28 01:32:50        0.0 /ifx_a/critical_files/sqlhosts/ifx_a.2
      
  12. For onbar to work, it needs an ixbar file (it´s own offline catalog). onsmsync command can recreate the ixbar file, but it can only do it from the sysutils database, which does not exists in Informix server ifx_b, since it has not been initialized (and even then, it would be empty). So we need to get it from the psm storage devices.

    • Get object 14 because it is the most recent backup of the ixbar file.
    • Check the object catalog metadata:

      $ onpsm -O detail -o 14
      
      Informix Primary Storage Manager Object Detail List
      
          Object ID       : 14
          XBSA PATH       : /ifx_a/critical_files/ixbar
          XBSA CopyId     : 0:14
          Object Type     : C
          Object version  : 2
          Physical pieces :
              https://ifx-a-cloud-bucket.s3.amazonaws.com//ifx_a/critical_files/ixbar/ixbar.14.1
          Number of pieces: 1
      
    • Get ("dump") the ixbar file from the storage device:

      onpsm -O dump -o 14
      
    • It creates a file ifx_a_critical_files_ixbar.14 on the current directory. Copy and rename the file to the become the ixbar file for the Informix server ifx_b (SERVERNUM 2):

      $ cp ifx_a_critical_files_ixbar.14 /opt/IBM/informix/12.10/etc/ixbar.2
      
  13. Launch the onbar restore command.

    $ onbar -r -w

    • Check onbar log file at /opt/IBM/informix/12.10/tmp/bar_act.log:

       2018-07-28 01:57:45.727270 11009  11007 /opt/IBM/informix/12.10/bin/onbar_d -r -w
       2018-07-28 01:57:45.794570 11009  11007 Using IBM's PSM version 12.10.FC12 as the Storage Manager. XBSA API version is 1.0.3.
       2018-07-28 01:57:45.799655 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:57:45.799874 11009  11007 Starting Filter /usr/bin/gzip -d -c.
       2018-07-28 01:57:45.831695 11011  11009 Successfully connected to Storage Manager.
       2018-07-28 01:57:46.435703 11009  11007 Begin reserved pages read (level 0 of rootdbs, Storage Manager copy ID: 0 12).
       2018-07-28 01:57:46.486687* 11011  11009 Filter closed.
       2018-07-28 01:57:46.531063* 11011  11009 Filter closed.
       2018-07-28 01:57:46.540890 11009  11007 Completed reserved pages read (level 0 of rootdbs).
       2018-07-28 01:57:46.555287 11009  11007 Unable to start logical log salvage: Unable to read boot strap config file - '/opt/IBM/informix/12.10/etc/oncfg_ifx_b.2' .
       2018-07-28 01:57:51.563908 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:57:51.564216 11009  11007 Starting Filter /usr/bin/gzip -d -c.
       2018-07-28 01:57:51.575587 11019  11009 Successfully connected to Storage Manager.
       2018-07-28 01:57:52.096184 11009  11007 Begin reserved pages read (level 0 of rootdbs, Storage Manager copy ID: 0 12).
       2018-07-28 01:57:52.153351* 11019  11009 Filter closed.
       2018-07-28 01:57:52.198363* 11019  11009 Filter closed.
       2018-07-28 01:57:52.209302 11009  11007 Completed reserved pages read (level 0 of rootdbs).
       2018-07-28 01:57:52.222134 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:57:52.222251 11009  11007 Starting Filter /usr/bin/gzip -d -c.
       2018-07-28 01:57:52.227977 11024  11009 Successfully connected to Storage Manager.
       2018-07-28 01:57:52.753398 11009  11007 Begin cold level 0 restore rootdbs (Storage Manager copy ID: 0 12).
       2018-07-28 01:58:05.396340 11024  11009 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 01:58:05.448496 11009  11007 Completed cold level 0 restore rootdbs.
       2018-07-28 01:58:05.790012 11009  11007 Completed whole system restore.
       2018-07-28 01:58:06.812501 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:58:06.812747 11009  11007 Starting Filter /usr/bin/gzip -d -c.
       2018-07-28 01:58:06.827447 11049  11009 Successfully connected to Storage Manager.
       2018-07-28 01:58:07.413841 11009  11007 Begin restore logical log 7 (Storage Manager copy ID: 0 13).
       2018-07-28 01:58:09.251399 11049  11009 The child process for the backup and restore filter is terminating
      with exit code 0.
       2018-07-28 01:58:09.382900 11009  11007 Completed restore logical log 7.
       2018-07-28 01:58:09.388014 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:58:09.831410 11009  11007 Completed logical restore.
       2018-07-28 01:58:10.046783 11009  11007 Successfully connected to Storage Manager.
       2018-07-28 01:58:10.104630 11009  11007 /opt/IBM/informix/12.10/bin/onbar_d complete, returning 0 (0x00)
      
    • Check online log file at /opt/IBM/informix/12.10/tmp/online.log:

      01:57:53  IBM Informix Dynamic Server Started.
      01:57:53  Requested shared memory segment size rounded from 4308KB to 4796KB
      
      Sat Jul 28 01:57:55 2018
      
      07/28/18 01:57:55  Requested shared memory segment size rounded from 110629KB to 110632KB
      07/28/18 01:57:55  Successfully added a bufferpool of page size 2K.
      
      07/28/18 01:58:00  Restartable Restore has been ENABLED
      07/28/18 01:58:00  Recovery Mode
      07/28/18 01:58:01  Listen thread init SUCCESS
      07/28/18 01:58:01  Physical Restore of rootdbs started.
      
      07/28/18 01:58:02  Chunk 1 '/opt/IBM/informix/12.10/tmp/demo_on.rootdbs' is sparse - inflating...
      07/28/18 01:58:09  Logical Recovery has reached the transaction cleanup phase.
      07/28/18 01:58:09  Logical Recovery Complete.
                8 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks
      
      07/28/18 01:58:09  Logical Recovery Complete.
      07/28/18 01:58:09  Quiescent Mode
      07/28/18 01:58:09  Checkpoint Completed:  duration was 0 seconds.
      07/28/18 01:58:09  Sat Jul 28 - loguniq 8, logpos 0x18, timestamp: 0x5f8dd Interval: 25
      
      07/28/18 01:58:09  Maximum server connections 0
      07/28/18 01:58:09  Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked 0, Plog used 19, Llog used 1
      
      07/28/18 01:58:10  Booting Language <spl> from module <>
      07/28/18 01:58:10  Loading Module <SPLNULL>
      07/28/18 01:58:10  Defragmenter cleaner thread now running
      07/28/18 01:58:10  Defragmenter cleaner thread cleaned:0 partitions
      
  14. The Informix Server ifx_b is now in Quiescent mode (memory is initialized, it can perform some administrative tasks). Change the Informix server ifx_b to online mode.

    •  $ onmode -m
      
    • Check that the database test_db was restored.

      printf "select name from sysdatabases;" | dbaccess sysmaster
      
      Database selected.
      
      name  sysmaster
      name  sysutils
      name  sysuser
      name  sysadmin
      name  test_db
      
      5 row(s) retrieved.
      
      Database closed.