Migrating Oracle E-Business Suite R12 from Linux 32-bit to Linux 64-bit

I try to document some of the issues we faced while migrating from 32 bit to 64 bit for application servers

1. The first misconception was that if we move to 64 bit from 32 bit, we would not be able to patch the application since there are not 64 bit patches available for all patches. There were conflicting opinions from Oracle Support (depends on the guys who pick your SR) that this is true  (you cannot patch 64 bit environments with 32 bit patches and  you would have to install 64 bit executables). This is not true and 32 bit patches can be applied on 64 bit platforms (this is confirmed in Steven Chan's blog too). In fact, Oracle probably does not specifically release 64 bit patches for Ebusiness suite.

2. Our Unix sysadmin did not install linux32 and this caused issues with all the shutdown and startup scripts. All of them check for the existence of linux32 before executing the stop and start commands. Exceptions are concurrent manager and apps listener startup/shutdown scripts.

3. Our Unix admin installed linux32 but we still had issues during startup. This was because he probably did not install it properly or got a wrong version. After installing linux32, we did not see issues with startup scripts but the command would just hang and timeout finally. 

This was confirmed by running these commands 

inux32

usage: linux32 [--3gb] [--4gb] program args ...

Default is --3gb to limit the address space of the 32bit children to 3GB

/u01/app/applmgr

applmgr@ofapdev21:OF1TCH> linux32 --4gb

Memory fault

/u01/app/applmgr

applmgr@ofapdev21:OF1TCH> linux32 --3gb

Memory fault

/u01/app/applmgr

applmgr@ofapdev21:OF1TCH> uname -m

x86_64

On a working system, where linux32 works properly the output should be like this

$ uname -m

x86_64

$ linux32

-sh-3.2$ uname -m

i686

-sh-3.2$ 

Our unix admin reinstalled linux32 and we stopped getting these errors anymore.

4. The other issue (this is silly and may be not everyone will face it) was rpc.statd process was not started on these application servers and we have a shared appl top (NAS) and even our INST_TOP was on NAS  (not shared between apps servers though). This caused some of the startup scripts not to acquire locks on files and these were seen in $LOG_HOME/ora/10.1.3/opmn/*.err files. Related issue was the update snaphot (through adadmin) never completed because it was waiting for acquiring a lock. I would see message like this

Trying to obtain a lock...

Deleting Temp Global Snapshot files ...Start time:Sun Oct 28 2012 18:05:39

Deleting Temp Global Snapshot Bugfixes ...Start time:Sun Oct 28 2012 18:05:39

Updating Global Snapshot files ...Start time:Sun Oct 28 2012 18:05:39

Total No. of records to be processed = 758374

5. The other thing that I noticed is that the startup/shutdown take considerably longer time on 64 bit boxes compared to 32 bit boxes (both have same memory). Still need to investigate this.

6. In the document " Migrating Oracle E-Business Suite R12 from Linux 32-bit to Linux 64-bit [Doc Id 471566.1]" one of the steps is Update CTXORIG.xml.

Instead of this why cant I directly update the context file to have this value

LINUX_X86-64 

and run autoconfig.

What is the difference between changing this in CONTEXT FILE vs ctxorig.xml. In fact, ctxorig.xml is generated by autoconfig and it would be overwritten even if I changed the ctxorig.xml and do not change context file and then run autoconfig

That was the question I had and for that Oracle had this response.

 I found you might find issues such as the one described in Note After Running Afcfgclone.Pl The Library Files On Apps Tier Disappear (Doc ID 564721.1) if you fail to set the right platform when performing the migration in the CTXORIG.xml. Per SystemCheck.java the check for the right platform is made first in CTXORIG.xml file.