SAP Basis Basics


SAP Basis Basics


I.   Welcome to SAP!

SAP was started by three IBM employees in 1972, and… blah, blah, blah… you can find this sort of information on the internet, in books and other publications, in SAP published documentation, etc.  This paper does not contain information which can be easily harvested from the internet but information from over a dozen years of SAP implementations.  Nor does it contain information to the other SAP implementation team – the functional team.  It is their role to have meetings, ask questions, create spreadsheets, and all the other tripe that makes them look really busy and valuable.

No, this paper is aimed at the technical side of the team – the Basis people – and takes a lighter view of things in the hopes that your already frazzled mind can stop screaming “Panic! Panic! Panic!” for at least a few minutes.  You are on the fringe of joining the Brother-and-Sisterhood of SAP Basis Professionals.  Of course, even that is now a misnomer since Basis is no longer called Basis – you won’t even find it in the SAP Glossary at http://help.sap.com anymore.  But we still do the same things.  So let’s start off by defining the Basis role:  anything having to do with the IT or technical part of SAP falls under the heading of Basis.  There are sub-Basis roles that might require specialist like Portals people, CRM middleware people, etc. but they still fall into the category of Basis. 

There is a old story about a project manager addressing his Basis staff:  “People – let’s solidify the role of the Basis team.  It goes like this:  anything I don’t understand about SAP is Basis.”  Sadly, on some projects, this is close to being true.

What does Basis mean?  I don’t know how the phrase was coined, but Basis functionality can be labeled the “non-flavored” base of any SAP instance which we will talk about in a minute.  Up until version 6.10, Basis was the only term for the “whatever” that was this base.  As of 6.10, Web Application Server – internet connectivity - was added to this “whatever” layer, so now you will see a few people described as being WAS consultants but it is all Basis.  If someone asks you what is your SAP area of expertise, you say you are a Basis specialist.  If you say you are a WAS specialist, people might just think that English is your 4th or 5th language.  And hardcore Basis people will point at you and laugh.

Besides, wait around a month or two and SAP will call it something different.

While we are on the topic of what to call things, here are the primary rules for SAPese.  If taken alone, SAP is not called sap.  It is always called S-A-P.  But when joined with other terms, you can call it sap – like SAPGui is sap-gooey, SAPScript is sap-script, SAPTech is sap-tech, etc.  If you call the standalone term SAP sap in front of anyone who works for SAP, you will get really nasty stares and possibly frantic little black notebook scribbling as your name and other personal information are logged for future… er, hmmm… use.


II.   What is IT’s role in a SAP implementation?  What can Your Implementation Consultants  do for Your IT Group?

It often seems that the Basis group does little while the Functional group has all these meetings and has to update all these spreadsheets and progress reports… but this is not true.  The Basis group has limited options when it comes to configuration – we pick the OS, platform, and database.  Once that is decided, we purchase the hardware and install the software.  At least part of this, the install of the first DEV instances, must be done by IT while the Functional group is still doing Blueprint so that they can use a SAP instance as a reference and demonstration resource.

And while there are some variables in picking the right OS, platform, and database, once it is done it is done and there is rarely any changing it later.  Some of those variables include which combinations have the largest SAP user base, which get newly released software the soonest,  which provide the highest and most stable, which receives the larget share of SAP R&D resources, which seems to guarantee the longest useful life, which allows the strengths of our existing staff to be put to the best use, which moves our company in the direction the entire enterprise needs to go?  So you do have to make some decisions.

The good news about this lack of choices on the part of IT is that we don’t have to produce nearly as much documentation as the Functional group.  This document is part of the Basis Information Kit.  Our main document produced using a SAP implmenetation is called the SAP Technical Infrastructure – STI Doc- document and it is a work in progress all through the SAP implementation.  Some parts of it, like the SAP instances to be installed, the clients to be created, the TMS procedures, etc. can be entered from the start but these procedures may change based on what works and what does not work for the Basis and Functional groups.  Other parts of the document must be completed later as they deal with server serial numbers, server DNS entries, SAP directory files, etc.  Generally, your implementation partner’s Basis lead will complete as much of the document as possible, and pass it on to the client so that the addition of new information is a continuing process.

The client Basis group will also receive some SAP publications and other Hitachi produced documents.  Besides the STI Document, they will be provided with a Basis Operations document tailored to your specific OS and database.  This document details approximately 95% of what a Basis person needs to do in order to handle the day-to-day chores of running a SAP instance.  The Basis Operations document will be reviewed jointly by the consultant Basis lead and the client Basis group.  The document you are now reading, Basis Basics, will provide a list of all Basis topics to be covered during Knowledge Transfer (KT) and daily, weekly, monthly, quarterly, bi-annual, and annual tasks your Basis group to schedule as appropriate, and an appendix containing practice exercises for the Basis group to do to get a hands on feel for the SAPGui interface.  Some SAP Tutor files, SIMs, may also be provided for use in teaching SAPGui navigation.  Your implementation partner may provide other SIMs and tutorial files as they become available.

Installation guides for all SAP software can be found at http://service.sap.com/instguides.  It is a good idea for the client  Basis group to download and distribute copies of the appropriate installation guides and scan through it if for no other reason that to become familiar with the terms SAP uses to reference IT entities, its software, and the SAP landscape.

Formal training by SAP is always encouraged, and it another good resource for an introduction on how SAP views the IT world.  The people you meet in these classes may be a valuable asset in the future if you run into problems with after your implementation partner is no longer enagged.  Having your company join ASUG – American SAP Users Group – is another good people networking tool and can be done at http://www.asug.com.


III.   What is a SAP Landscape?  What is my role in a SAP implementation?

The SAP Landscape is like a layout of a complex garden – you have areas for roses, and areas for lilacs, and it is all laid out in proper form.  A SAP landscape can range from one SAP instance with one client and one user who does all the input into the instance via keyboard to dozens of instances with hundreds of clients and thousands of users, with keyboard input, RFC (Remote Function Calls) and ALE (Automatic Link Exchange) from other SAP instances, links to external databases, EDI (Electronic Data Interchange) via flat file from banks, vendors, etc., and RF units in the warehouse.  In other words, a SAP landscape can be very simple, or very complex.  A bed of petunias, or the Hanging Gardens of Babylon.

A normal SAP landscape consists of a Development (DEV) instance, a Quality Assurance (QAS) or Test (TST) instance, and a Production instance (PRD).  Some very small implementations will have only a DEV and PRD instance, with the DEV instance containing a QAS client for testing purposes.

A non-Production SAP server should have 2 (preferably more) processors, at least 4 gig of memory, and at least 100g of disk space.  A Production SAP server should have at least 4 processors, from 4 to 8 gig of memory, and at least 200g of disk space.  A hefty server with 6 – 8 processors, 6 – 8 gig of memory, and at lease 200g of disk space can host two SAP instances.  This can be done for DEV and QAS but PRD should never share a server with any other SAP instance.

The SAP instances form the core of a SAP landscape.  The other installed SAP products are peripheral to the SAP instances.


IV.   What is a SAP Instance?

A SAP instance is all the components created by the SAPinst program who all share the same database.  There are three mandatory sub-instances;  the Database Instance (DB), the Central Instance (CI), and the Dialog Instance (DI) aka Application Server.  This is the minimum configuration of any SAP instance.  There can be multiple DIs but only one DB and only one Active CI which means that a copy of the CI can exist for High Availability but only one of the CIs can be active at any given time.  You will hear about SAP being multi-tiered and that term is referancing these three layers.  Sometimes you might see “other SAP tiers” like ITS but that really isn’t a separate tier, it used to be an additional piece of software working with IIS, and now ITS is part of Basis/WAS so it is part of the CI Tier.  These layers, plus other SAP written software, are also known as the SAP Business Framework.

You can probably guess at what the DB contains.  There are any number of tables in a SAP instance, from 21,000 to 38,000.  Many have four or five character names that are usually abbreviations of things like USR for user, MANDT for client, etc.  You could guess that USR is short for user.  But MANDT?  So, we add one more variable to the equation – those four or five character names are abbreviations of German words.  Needless to say, trying to look at SAP from a typical DBA prospective is almost impossible.  Fortunately, SAP supplies the tools for you to manage all these tables.

The Central Instance is a lump term for all the SAP executables, the installed OS file structure, and anything that is placed on the OS to support and communicate with the SAP instance.  It “talks” to the database, handles requests made by the application server(s), and sends back the information.  Other software products often call this the middleware layer.

The Dialog Instance connects the users to the CI, passes the issued requests to the CI, and sends the returned results to the user’s session.  SAP uses a client piece called SAPGui which handles the user-to-DI communication on the user’s workstation.

These are the three main pieces of a SAP instance installation.  There are other parts that can be added for various sub-access and external tasks.  For a Development SAP instance or a Quality & Assurance or Test SAP instance, all three layers are normally installed on the same server.  For a Production SAP instance, the DB/CI are often installed on one server and the DI on another server for load balancing purposes.  It should be noted that the installation of a CI instance automatically installs a DI which can be used by everyone if needed, be used only as needed by Basis staff, or never be used.

Instance can be a difficult term to understand – almost every major database applies this term to the installation of the database software.  So if you see reference to an Oracle instance, this means the installation of the Oracle software.  An Oracle database is the creation of a new empty database within that Oracle instance.

Understanding the Startup and Shutdown prodedures may help solidify this layer concept.

The normal SAP instance start up consists of three parts: starting the SAP OS Collector, starting the Oracle Listener, and starting the SAP instance. The process mainly goes like this:  ora<sid> logs on and starts the Oracle Listener then  <sid>adm logs on and runs the startsap script.

What?  You say we missed a step?  What happened to the SAP OS Collector?

The startsap script takes care of the SAP OS Collector for us.  When the SAP Instance starts up via the startsap script, it checks to see if saposcol is up and running – whether from the root user starting it manually or from another SAP Instance already starting it up, it doesn’t matter.  If saposcol is up and running, the script simply moves on to the next step.  If it is not, the script starts saposcol as root and then proceeds. So the SAP OS Collector gets handled one way or another.

However, you may need to bring up multiple Oracle Listeners depending on the database configuration.  If the MCOD installation option was used then only one Oracle Listener is used since both databases share one Oracle listening port which is normally 1527.  If, however, two Oracle instances were installed, and each database uses its own unique Oracle listening port, then multiple listeners must be started.  The startup procedure for each SAP Instance would be exactly the same as if only one SAP Instance resided on the server.

The process to stop the SAP instance is close to being the reverse of the start procedure.  <sid>adm stops the SAP Instance, ora<sid> stops the Oracle Listener, and root stops the SAP OS Collector.  The only real difference is that saposcol is not automatically stopped by the stopsap script – there may be other SAP instances on the server which means this software needs to stay up and running to gather OS information until that instance comes down.

One thing to note – the Oracle Listener does not start the database, it simply “watches” port 1527 for any database related activity.  The startsap and stopsap scripts handling the startup, mount, opening, and shutdown of the database.  None of this activity could occur if the listener was not “polling” port 1527, relaying the requested database function, and returning the results through the same port.

The <sid>adm and ora<sid> users are assigned environment variables using the SAP installation run that identify them as users of a specific Oracle database and SAP instance.  So when oraabc starts the lsnrctl program with the “lsnrctl start” command, oraabc’s environment variables tell the server which database for which the listener is to “listen”.


V.   What are the different kinds of SAP software?  What kind of hardware will they need?

Although the most common flavor is SAP is called R/3.  This is SAP’s main ERP product that handles Financials, Controlling, Logistics, Sales, Human Resources, and lots and lots of other ERP stuff.  All these modules – that is what the functional people call them – live in and use common data in one SAP instance.  All the modules are installed during a SAP instance installation.  Whether all the modules are implemented is up to the SAP customer.

Each SAP “flavor” must be installed in its own SAP instance.  Products such as APO (Advanced Planning & Optimization), BW (Business Warehouse), CRM (Customer Relationship Management), SRM (Supplier Relationship Management), SolMan (Solution Manager), and SCM (Supply Chain Management) are all SAP “flavors” installed into their own instances all of which are pretty much installed the same way – installed with a DB, CI and DI - and the process is pretty much the same depending on the age of the product.  Just like SAP’s product line, its installation program SAPinst has changed over the years as well.

In addtion, there are add-ons supplied by SAP.  Products such as plug-ins for communicating with other SAP instances; the SAP Learning Solution, Sarbannes-Oxley, Best Practices, etc. can be added to an existing SAP instance.  These add-on products are what the name implies – they “add” more functions and data “on” to an regular SAP instances.

Some SAP transactions communicate with external software but the functionality is either in the SAP instance or out on your network or on the internet.  SAP transactions like SCOT use built-in communication ports for the use of sending email from SAP to a SMTP server.  Transaction SICF activates business packages for access via the internet. 

RFC – Remote Function Calls – can originate in the SAP instance and go out to “talk” to some other product that knows what to do with the request passed, and information is send back into the SAP instance.  And, of course, an SAP instance can talk to another SAP instance using RFC – for example, this is how the Transpost Management System moves changed from one instance to another.  There are also products written by SAP partners which can be accessed externally from a SAP instance:  products like Vertex which is a Sales Tax database, and BSI which calculates payroll taxes.

As for hardware, SAP has special “hardware sizing” software on which it trains it’s hardware partners like IBM, HP, Dell, etc.  They have worked together to create templates for very small, small, medium, large, and very large SAP landscapes.  Once you contact the hardware vendor of your choice, they will meet with you to calculate your SAPs – a unit of measurement SAP uses to estimate workload – decide into which category your company fits.  From there it is pretty much a plug-in the company data and spit out a sizing reports.

Let’s get this out into the open early:  a Basis consultant does not do hardware sizing.  Normally, they will look at the sizing report from your hardware vendor and tell you if they think it is too weak or too string.  That is about all you can expect of them – their career could be very short-lived if they say “Hey, they are just trying to get you to purchase some really expensive boxes that you just don’t need!”.  Remember, SAP created the sizing program and trained this hardware vendor on it and that is what your quotes will be based.  Anything critical said about the proposal could get back to all sorts of people who could more or less hurt you in certain circles.  I’m not talking about punching you out or stalking you.  More along the lines of every problem message you open with SAP gets its priority bumped down to like a -5. 

Also, we recommend that you get hardware estimates from more than one hardware vendor.  Even if you mentally are locked into an IBM box running AIX and Oracle, it doesn’t hurt to see what the other platforms are selling for.  And it doesn’t hurt if rumor gets back to IBM that you are talking to Dell.  Playing hardware vendors against one another can lead to some hefty discounts given by a vendor who doesn’t want to see a sale walk out the door.

You can get a head start on the hardware sizing process by reading the documents available at http://service.sap.com/sizing.  If nothing else, you can impress your hardware vendor by sprouting out all these SAP terms like SAPs and High Availability and SANs. 

Besides servers for your database and SAP instances, you might need other servers for your SAP landscape depending on the complexity of your SAP implementation.  At the very least, you will need an additional not-too-hefty server for a Solution Manager instance which is what SAP is now requiring as the “tunnel” to get into your SAP landscape; you’ll need a small server having a public IP address and located in your DMZ for your OSS connection.  More about this later.

And there may be a need for Portals servers, UME server, a SAPConsole server… the list goes on and on.  As stated earlier, a SAP landscape can be very simple or very complex.

Remote installation on your SAP servers can make the installation task simpler – it is always more comfortable to do the work from your own workstation than having to sit in a cold, noisy computer room.  The Basis group of Hitachi recommends Terminal Services – aka Remote Desktop – for this work.  PCAnywhere and products such as Tight VNC do not possess the stability of Terminal Services.


VI.   How does SAP get installed?

First, it must be noted that only the SAP company is allowed to KT (Knowledge Transfer) installation information.  Clients may watch all they wish during installation, and are encouraged to install and delete sandbox instances on a spare server rather than let it gather dust.  But installation is not part of your purchased Technical Knowledge Transfer package.

Without getting into the little details, here is what gets done during the typical SAP install – the work that gets done after the hardware has been installed into the network and DNS:

1.    Make all server configuration adjustments as specified in the SAP installation guide.
2.    Download and install JDK 1.4x – you can use JRE 1.4.x if you aren’t going to use the ABAP add-on.
3.    Install the database software.
4.    Patch the database software.
5.    Run SAPinst to install the CI and default DI instances.
6.    Run SAPinst to install the DB instance.
7.    Run SAPinst to install any additional DI instances.
8.    Download and install the most current SAP kernel.
9.    Current the TMS for the SAP instance.
10.  Download and apply all outstanding SAP patches.
11.  Request and apply the permanent license key.
12.  Do remaining post-installation work such as connecting SAP Online Help, creating clients, adding users, etc.

Normally the post installation work takes 2 - 4x the amount of time of the actual installation.  The installation itself normally take a one eight hour working day and the post-installation work two eight-hour working days for a total of three working days to install a SAP instance.  This regardless if it is a DEV, QAS, or PRD instance.


VII.  Do I have to install JDK 1.4x?

SAPinst – the SAP installation program – needs at least JRE 1.4x in order to run the SAPinst interface when you install a new SAP instance.  But if you intend to add the Java Add-on or Java components after the SAP instance is installed, you need JDK in order to do so.


VIII.   What is a SAP client?

OK, we have the concept of a SAP instance, and that instance has a database which contains thousands of tables which contain a whole bunch of rows.  After SAP is installed, these tables need to have some base data in order for customization and configuration to begin.  Like state abbreviations, country codes, HR titles, etc.

SAP provides a subset of this data so that the Basis team can get in the new instance, add themselves a user ID in client 000 – “our” client - and start the real work.  We don’t want to mess this subset of data up so we need to populate it to a “work” place for the Functional Team to do their work.  Or several work places.

The base SAP instance comes with two clients:  000 and 066.  Forget client 066, it is used by SAP when you get close to GoLive and you want an EarlyWatch report.  This is optional and I believe a fee is involved for this service.

All the base or subset data is contained in client 000.  Also, client 000 is where the Basis Team does a lot of its maintenance like patching.  The Basis Team people are the only implementation members who will ever have access to client 000.  You can think of client 000 as the owner of all the client independent data in the SAP instance.  Explanation in a minute.

So, in order to let the Functional Team do their own thing without screwing anything up, we create a new client for them.  Think of a client as a view of the database.  If you log into client 000, you can see all client independent data – like the ABAP programs – and all the data that is dependent on client 000 only.  If you log into client 100, you see all client independent data – like the ABAP programs – and all the data that is dependent on client 100.  You can’t see the data that is dependent on client 110 while you are logged on to client 100.

Thus the concept of client dependent and client independent data.  All rows in some tables are accessible from any client like T000, the Data Dictionary tables, tables that contain the ABAP programs, printers, etc.  These are said to be client independent.  Data like users, companies, vendors, customers, etc. are client dependent.  You have to go into a specific client in order to see this data.


IX.   How do I create a new client?

This is a pretty easy procedure.  Basically, you add a new logical system in transaction SALE, create a new client in transaction SCC4 using the logical system you created previously, create a RFC target destination using transaction SM59 with the same name of the logical name, create a RFC source using transaction SM59 for the source or “from” client, log on to the new client, and schedule a client copy from the source client to the destination or “to” client.  That’s all there is do it

When you bring up a new SAP instance, you add a client 100 and schedule a client copy from client 000 to client 100.  This is called a local client copy since the source or “from” client is contained within the same SAP instance as the target or “to” client.  When you add a new SAP instance to your landscape, like QAS, you might want to copy client 100 in the DEV instance to client 200 in the QAS instance.  This would be a remote client copy.

A client copy is destructive – in other words, all the data in the target client is deleted during the client copy.  So the procedure is not just for creating new clients but refreshing existing data.  The only exception to this rule is the using the SAP_CUST profile to do the copy – it will leave all the data in the target client intact with the exception of the user master data which will be deleted and replaced with the source client’s user master data.  You can even mix and match, the data from one client and the user data from another, and copying them both at the same time in the same client copy run.

A client is created using transaction SCC4 and at the time of creation you must specify what type of client it will be.  Is it to be used to create configuration changes that are to be transported to QAS and PRD?  Is it a reference client, “frozen” so it can be used to refresh the data contained in test clients?  Can both client-dependent and client-independent data be changed in this client, or only client-dependent data, or no changes allowed at all?  These various types of client have their own labels to the SAP implementation team:  golden client, unit test client, configuration client, ABAP client, production client, etc.  If someone references a client with which you are not familiar, be sure to ask for clarification so that the wrong client does not end up being the source client for a client copy!


X.   What is this permanent key stuff?

When a SAP instance is first installed, it installs with a temporary license that expires in 4 weeks.  You must request a permanent license key via the SAP Marketplace.  More about that later.


XI.   How do the users communicate with the SAP instance?

Well, of course, you need to add them to the SAP instance.  Then the user has to be able to connect to the application layer in some from.  Normally this is using the SAP supplied program SAPGui which is installed on a user’s workstation, but here are other ways such as connecting to a Citrix server on which SAP is installed or connecting from the internet using a SAP product via Portals, ITS, or some other product, or using one of the SAGUi alternatives like SAPGui for Java.

SAP uses some common ports no matter where it has been installed.  The first instance installed on a server is labeled as System Number 00.  It would use ports 3600 for the Dispatcher, 3300 for the Gateway, and 3600 for the Message Server.  If you had a second SAP instance installed on the same SAP server, it would normally use the next available number in the port range.  So SAP instance #2 would be System Number 01, use port 3601 for the Dispatcher, 3301 for the Gateway, and 3601 for the Message Server.  As long as a user can access these ports from his workstation, he can log on via the SAPGui installed on his workstation.


XII.   How do I limit what my users can do when they log on to a SAP instance?

SAP user security is done via roles.  A role is basically a group of transactions, authorizations, and authorization objects bundled together for use by a group of users with common functional needs.  This group definition is saved and generates a profile which is attached to the SAP user who needs it. 

People often get intimidated by SAP security – there are thousands of transactions, authorizations, and objects.  It helps to think of all this is simplistic terms.

Let’s say you have a user who needs access to one transaction in the SAP instance.  All he needs to do it log on, go to transaction SPAD, and print a list of available printers.  That is his whole job, to create this printer list once a day.  So he needs the authorizations that allow him to log on to SAP, to go to the SPAD transaction, and to print the list to any available printer.  He has * to all printers.  He basically needs a role containing the transaction he is allowed to do, SPAD, and the authorization object S_SPO_DEV set to all because he can see and print to any printer.

Then let’s says a new check printer gets added to SAP, and we don’t want the user to be able to print his printer list to the new check printer.  So we change authorization object S_SPO_DEV to a list containing authorizations for the printers to which he can print, excluding the check printer.  His dropdown list for printers won’t even show the new printer now.  And he will be able to print to any printer in his printer authorizations list.

This is a simplistic example but you can see how it works.  SAP security can be very tight or very loose.  In general, it is loose in DEV, using a modification of the SAP_ALL profile – sort of God-like powers and what the Basis people normally use since it already exists and why make a role that does the very same thing? - to allow every DEV user to do everything with the exception of Basis tasks.  QAS may be a little tighter, or a combination, allowing the user SAP_ALL during training but limiting the user to his/her PRD role in another client on the last day so he can test the role he will really use out.  In PRD, of course, users should only receive the rights to only what they absolutely need to do.


XIII.   How do I patch a SAP instance?  How often should I do it?

There are three different types of a SAP patch:  support packages, kernel replacements, and SAP Notes – aka OSS Notes.

Support packages are used to make ABAP code and functionality changes.  ABAP is the language in which SAP is written and it is pronounced AAAA-bop and not AAAAA-bap.  Support packages contain a group of ABAP “fixes” that must be installed via transaction SPAM from within a SAP instance.  You apply these “fixes” in a specific order:  Basis, ABA (Application Basis patches – the software that supports the interaction of SAPGui with the SAP instance), APPL (application functional patches), and then plug-ins, etc. that also exist in the instance because add-ons and stuff have to be patched too.  After you apply support packages to a SAP instance, you need to regenerate all the ABAP loads so that users don’t have to sit watching Compiling… messages the first time a changed ABAP program is called.

Kernel patches are simply a replacement of the SAP executables on the OS level.  This is always done to a new SAP instance and most likely done following a support package application run.

SAP Notes – or OSS Notes – contain a patch or two to solve an immediate problem.  The Basis staff either manually applies an ABAP code fix or uses the SNOTE transaction in the SAP instance to apply the fixes.  There are times, however, when the note contains instructions that must be done manually – in other words, things must be done that SNOTE cannot do.  Eventually, as the SAP Note corrections become more and more, SAP will group them together and release a new support package so that they can be applied en masse and not one-by-one.

In order to download any of these patches, you must log in and download them from the SAP Marketplace at http://service.sap.com/patches.  And you must have a valid OSS ID – aka SAP Front-end User ID or “s” number – with the rights to access the patches webpage.

Other products, such as J2EE, are patched via a combination of deployment of packages and replacement of OS executables.

In general, you should schedule your support package maintenance often enough to prevent shell-shock to users due to massive changes in the SAPGui screens, and rarely enough that you don’t spend all your time preparing for the next support package application window.  Often, patch application is driven by an identified problem that can be fixed via a support package referenced by SAP.

Just make sure to get a list of all the packages that will be applied, and use the SAP Notes Side-Effects tool off the SAP Support Packages in Detail site at http://service.sap.com/patches to get a list of everything that is changed during patching for your Functional team leads.


XIV.   What are all these numbers?  “s” number, license number, etc.

When you become a new SAP customer, SAP assigns you a customer number.  This number is like any other customer number you assign your own customers, or that companies assign to you.  It uniquely identifies your company to SAP.

Once you sign your software license, each of the SAP components you purchase is assign an installation number. So, you signed a mySAP.com Business Suite license?  Then you will be assigned an installation number for R/3, another installation number for BW, another installation number for CRM, etc.  You also signed a license for KW?  That would be another installation number.  SAP sometimes splits existing products out to other divisions, so you may even be assigned a second customer number with new installation numbers assigned under that number.  SAP did that with Enterprise Portals.  An installation number identifies a customer's SAP component.  If you open a problem under your R/3 installation number, SAP knows that the problem is going to deal with R/3 and not CRM, BW, etc.

A few weeks later, SAP shipped your installation kits and you installed R/3.  Four weeks later your users get a message saying that the license has expired.  But you signed your license agreement!  When an SAP instance is installed, it gets assigned a temporary license key that is good for approximately four weeks.  You need to request a permanent license key as soon as you finish your installation of the SAP instance.  How?  First, you will need your hardware key.  This identifies your operating system information so SAP can generate a key that is not only good for your SAP version but for your hardware as well.  To see your hardware key, log on as <sid>adm and go to a command prompt.  Type "saplicense -get" and your hardware key will be displayed.  Jot it down, you will need it later. The hardware key for NT is different than, for example, the hardware key for AIX.  So if you change your SAP license from AIX to NT, you will need to make sure that your company is attached to the new hardware key or you won't be able to request new permanent license keys.

To request your new permanent license key, go to http://service.sap.com/licensekey.   Use this website to give SAP information on which SAP flavor your have installed.  You will have to provide your hardware key as well.  SAP will generate the new key and e-mail you a text file.  Save the text file on your SAP server, log on as <sid>adm, and go to a command prompt.  Type "saplicense -install ifile=<path and name of txt file from SAP>".  Your key should be installed!  In later versions of R/3, you can use the SLICENSE transaction as well.

Now your system is flying!  But your ABAPers sign on for the first time, and they need to add a new program to upload master data.  When they enter se38 they get a message asking for their developer key.  This key is generated by SAP as well, and is used to register your super users who will be responsible for adding new code and changing SAP-owned code.  Go to http://service.sap.com/sscr to register your programmers.  Once you have the developer key, give it to your ABAPer and he can enter it into the system.  He will only have to supply it once per SAP instance.

Now your ABAPers are cranking out code, your system is humming, and then!  Pow!  The dreaded short dump hits you when you aren't looking.  You look up the error code and find a valid OSS note to fix the problem.  You log on to apply the advance correction, your give yourself a developer key, go to transaction se37, but a new popup won't go away, it keeps asking for an access key for R3TR  FUGR XXXXXXXXXXXXXXXXX.  You must get an object key in order to modify an SAP-owned object.  Since SAP owns all the ABAP code that comes with the SAP instance, you have to get a key in order to apply the advance correction.  Again, you would use http://service.sap.com/sscr, this time to generate an object key.

All this SAP website access also demands that you have an OSS User ID, or as it is now known, SAP Service Marketplace User ID or “S” Number.  The user ID to perform the tasks outlined in this message must have administration rights.  You should have received your primary OSS ID when you got your SAP license.  If you have not received a primary OSS ID, or your SAP reseller seems to have control of your primary OSS ID, contact SAP AG to resolve the problem.  Your OSS ID is connected to your SAP customer number, so if you have two customer numbers, you will receive two OSS IDs.   Make sure to limit the powers of any subsequent OSS ID’s generated by the Basis group.  More on this later.

SAP has recently complicated the license scenario by introducing user licenses.  Do not confuse your software license keys with SAP licensing of the usage of named users.  These are two different things!  Usage by users of an SAP instance will be monitored by your auditing team, and normally the only function of the Basis group in this regard is using su01 to register the "type" of each user.

Now there is only one other really important number you need to know:  the phone number for your Basis technical consultant!


XV.   How do I add users?  Do I have to add them all to each client one-by-one?

You use transaction SU01 to add and change users.  If you want to copy the users from client 100 to the users in client 110, you can do a special kind of client copy that is non-destructive.  When the profile SAP_USER is used during a client copy, just the user master data is copied from the source client over the target client, and the rest of the database is left intact.

No matter how you manage your user security, there is no getting around the fact that you have to manually add them all to at least client at some time or another.  The best way is to add them all to client 100 in DEV since that is the “Golden” client and should never be “refreshed” or written over by a client copy.  Once you have them all in one spot, you can use a client copy for just user master data and pop all the clients wherever you want them.

There is another way to add users to clients called Central User Administration (CUA).  CUA allows you to designate one client in the SAP landscape that is the parent of all or specified clients, both local and remote.  When you add a new user to the parent client, the child clients get populated with this data.  Thus, one user add will trigger others within the instance and to other SAP instances as well.  This technique should only be used on clients that are not to be refreshed in the future.


XVI.  Do I have to add printers to each client too?

No, printers are client independent data.  Once you add a printer using the SPAD transaction in an instance, it is available to all clients in that instance.


XVII.   So what does the Functional Team do in their own little client(s)?

They do configuration and customization.  Think of configuration equal to you going into Outlook on your workstation and changing things in your Tools -> Options.  You change your Calendar Default Reminder to 5 minutes instead of the default 15 minutes.  Now, multiple that by about 1000x and you have an idea of what configuration entails.  This configuration covers tax rules, country codes, company codes, sales regions, sales areas, sales pipeline, all that stuff technical people really don’t want to know.

Customization is changes made to SAP-owned objects when configuration does not have an option for what you need to do.  For example, SAP comes with several canned Sales Invoice documents.  If none of them contain a format you can use, your Functional Team will work with an ABAP consultant to create a new Sales Invoice document that better suits your company’s needs.

Depending on the configuration and customization changes made, the SAP instance may force you to register these changes in the form of transport.  The implementation will try to group all these changes into transports that contain all the work done for a project or a unit of work so that they can be moved to QAS and later PRD easily.

The TMS (Transport Management System) is used to move these transports between SAP instances.  Movement of these changes between clients in the DEV instance can be done by the functional people using SCC1.  What goes on in the DEV instance mostly never is important to Basis.  Once the changes need to go to QAS, the Basis team takes over and uses TMS to move them for the functional team. 

TMS movement can be very loose or very tight.  Some shops simply release a transport and then send email to the Basis group asking them to move the transport into QAS, and later into PRD during off hours.  This is called manual TMS control.

Some companies schedule jobs to move any transport found in the QAS queue to QAS at a set time once or twice a day.  The transport is also moved into the PRD queue where is stays awaiting approval.  Once the transport has been tested in QAS and is ready for PRD, the project supervisor and/or system administrator uses a transaction to OK the change import into PRD which releases the transport in the PRD queue which gets imported into PRD during the next scheduled PRD TMS job run.  The method is automatic and requires little intervention by the Basis Team.

Most companies use some combination of these methods.  There are several variations such as moving changes to client groups in DEV the same time the change is imported into QAS, and importing the change into multiple clients across the landscape.

The TMS tool is used by the SPAM and SAINT transactions in order to manage the support packages waiting to be applied which is why it is one of the first post-installation steps that must be done in a new installation.  All the SAP instances of a particular flavor share a transport directory which is normally owned by the DEV instance since it is the first instance installed.  Different SAP flavors need their own SAP transport directory, meaning all CRM instances share a TMS, all BW instances share a TMS, etc.  Very few transports can safely be cross-instance applied since not only the SAP flavor should be the same but the Basis version as well. 

ABAP programmers are “special”.  No only do they create new ABAP programs, they sometimes have to change existing ABAP program.  These programs are SAP-owned so things can get a little sticky.  Whether adding a new program or changing an old one, every programmer need a developer’s key that we previously discussed.  This will be fine for new programs, programs that should either begin with “Y” or “Z”.  So if you hear someone reference Z programs, you know what they are talking about, it is a general term to refer to all “home-grown” SAP programs and include not only the programs whose names begin with the letter “Z” but the ones that being with “Y” as well.

SAP doesn’t like programmers just going in and changing ABAP code that it “owns”.  And remember, ABAP programs  are client independent, so if a bug gets into a program from a change made by a programmer in client 120, it affects client 000, 100, 110, etc., all clients share one version of the ABAP programmer.  As a form of security and to allow better tracking of SAP-owned objects, a key must be generated at http://service.sap.com/sscr - SAP Software Change Registration – that is unique to the object about to be changed.  Only the Basis group can do this, programmers cannot request SSCR key on their own, or that is the way your Basis group should make it work.  Basis has control over the other “S” numbers generated against your SAP license, and you have to have a “S” number with administration rights in order to generate a Developer’s Key or an Object Key.  So, genereally, when adding a new “S” number for a non-Basis group person, allow note searching and any message to SAP options “on”, and everything else “off”.


XVIII.   What factors affect SAP Performance?

Like every other complex piece of software, performance in a SAP instance depends on many factors.  So you often have to play detective to find the core of your problem. 

The most common complaint is when a SAP instance is cycled – meaning stopped and restarted.  As we touched on earlier, a SAP instance is comprised of millions of lines of ABAP code which are generated into load objects.  When a SAP instance is stopped and restarted, all data and program information is flushed from SAP’s internal buffers.  So every time a new SAP program needs to run after being cycled, the needed program has to be read and then placed in the program buffers for use.  Not only that, if the ABAP code for that program has been changed or the load object invalidated in any way – like a kernel replacement that added new functionality, etc - the program must FIRST be recompiled and then loaded into the buffer.  The next time the program is called, it will execute faster since its load object is already out there in the buffer.  The most commonly used programs in a SAP instance number in the thousands, so as the day goes by, SAP response time goes faster since getting a “hit” in the buffer for a certain program happens more and more often.  But there might come a time when some of these “fast” program have to be switched out of the buffer because their date/timestamp is the oldest of all the programs in the buffer, and the buffer is full.  Buffer swapping can slow a SAP instance down.

Another factor that often causes a slow down or even a hanging SAP instance is failure to “dump” the database log.  This occurs more often in Oracle instances since a MS SQL Server database will just keep growing until there is no more room on the hard drive, which is something rather easy to spot.  Many people fail to process the ReDo files created by an Oracle database by backing them up to tape and informing BRBACKUP that they have been processed, and they just keep accumulating until there is no place else to put them.  This can cause the database to hang which causes the SAP instance to hang.  For a DEV or QAS instance, most DBAs turn “off” log archiving.  On PRD instances where a backup should occur nightly, a task can be scheduled to delete all ReDo files older than 2 or 3 days as long as the archive files are backed up when the rest of the SAP instance is backed up.  Another common problem with Oracle is if the logging and automatic archiving are turned “off”.  Both should be never both be off to prevent the archiver from getting “stuck”.

It is possible for someone or something to get into an infinite loop and just keep eating up your CPU.  Transaction SM50 shows you what is running at any given moment.  Almost every process shown in SM50 has a corresponding process on the server side.  You can use the process ID in SM50 to investigate the corresponding tasks on the server.  You can normally use SM50 to cancel any “hung” processes but sometimes you have to go to the OS level and cancel the process there.

Server space is not just critical on the hard disk where the logs are created.  Any shortage of hard disk space on a SAP server should be corrected at once.  SAP is like any other software; it uses TEMP space, and generates other type logs, like job logs and print logs.  You want to make sure there is plenty of space available on all your SAP server drives.

Index rebuilds and statistics jobs should be scheduled on all your SAP instances on a monthly basis.  This will keep your database tuned and efficient.  Any time you do a mass program regen using transaction SGEN, you should schedule a statistics run on the database as well.

Every SAP instance contains three profiles that let us tune and adjust various parameters such as buffer size, default client, number of processes, etc.  Rarely will you touch two of these, the DEFAULT.PFL and START_DVEBMGS00_<server>.  The third, <SID>_DVEBMGS00_<server>, you will probably spend the rest of your SAP career changing.   Erroneously changing one of these profiles is common reason why a SAP instance that was working perfectly in the past won’t come up now.  That is why even though you make your changes to the profile using transaction RZ10, once you activate it, it gets written as a flat file out on your instance server.  So if a parameter change causes your SAP to fail during startup, you can undo the changes with VI on UNIX or notepad on Windows.


XIX.   What the heck is all that DVEBMGS00 crap?

Up to now, we have discussed that a SAP instance contains three components; the DB, CI, and DI instances.  But what do I SEE on the OS level when my SAP instance is up and running?  The answer to this is complex, and you aren’t even close to being able to understand it all, so we will just present the big picture for now.

Besides having your database software running, and any shadow processes it might spawn, your SAP instance creates many processes when it successfully comes up.  It may be configured to start at least one of every type of process SAP uses during the average day, or it may be a more specialized SAP instance that just handles user communication, leaving the more muscular work for the CI elsewhere.  But every SAP instance will have at least 1 work process kicked off when it starts.

The types of work processes are as follows:

     Type                        Description                                                      Process                                                Instance Parameter

     Dialog          Process user requests in foreground                    dw.sap<SID>_<DVEBM>                        rdisp/wp_no_dia
     Update         Server uses to process mission critical updates     also dw.sap<SID>_<DVEBM>     rdisp/wp_no_vb
     Update2       Server uses to process less critical updates                      also dw.sap<SID>_<DVEBM>     rdisp/wp_no_vb2
     Enqueue       Used for lock management                                  also dw.sap<SID>_<DVEBM>     rdisp/wp_no_enq
     Batch           Process jobs in the background                           also dw.sap<SID>_<DVEBM>     rdisp/wp_no_btc
     Message       Message server                                                  ms.sap<SID>_<DVEBM>                       N/A
     Gateway       Gateway server                                                 gwrd                                         N/A
     Spool           Used for spool management                                also dw.sap<SID>_<DVEBM>     rdisp/wp_no_spo

So, given that the German word for update starts with a V, the standard SAP instance that is the first instance on a server – i.e. System Number 00 – you get DVEBMGS00.  So I can tell just by looking at the \usr\sap\<SID>\DVEBMGS00 directory on the OS level that this is a full SAP instance – it has all the parts.  A SAP instance can only have 1 message server, so we know this is the main directory for the instance.  When a DI is optionally installed, the name is normally in the form of D<System Number> since it is subservient to the main SAP instance.

Just looking at the processes running on a server, you can pretty much tell if your SAP instance came up sucessfully or not.  First, the message server comes up, and then the gateway/dispatcher starts and connects to the message server.  And finally, the disp+work work threads for the dialogs, batch, and update processes start, the enqueue and spooling specific ones first.  One depends on the other, the dw’s can’t start unless the gateway/dispatcher is up which can’t start unless the message server has come up successfully.  The reverse is true during instance shutdown, first the dw’s go then the gateway/dispatcher, and finally the message server shutdown will always be the last thing in your log when you stop your SAP instance, it isn’t down until that messsage shows in the log.


XX.   And this <SID> thing?

SID stands for System Identifier or something pretty similar.  Each SAP instance must have a three character code that is unique on any given server, and should be unique within any given SAP landscape.  That is where DEV, QAS, TST, and PRD came from.  Anytime you see a reference to <SID>, it takes the place of the SAP instance’s system identifier in upper case, <sid> is the same in lower case.

DEV, QAS, TST, and PRD used to be good enough to handle your entire landscape.  But with new SAP flavors and their instances came the need to modify this slightly.  Different implementations do things differently but they should implement a plan that can grow as their landscape grows.  Generally, we use the first 2 characters to designate the SAP flavor, and the last the part of SAP instance.  So the R/3 instances would be R3D, R3Q, R3T, R3P, and maybe even a R3S installed at some point or another as a Sandbox.  The same for CRM – CRD, CRQ, CRT, and CRP.  I think you probably get it now.

When a SAP instance is installed, its directory structure will look like this:

          usr ->
                        sap  ->
                                    <SID>  ->
                                                DVEBMGS00  ->
                                                            data
                                                            exe
                                                            igs                                * IGS executables
                                                            log                               * Audit and Trace logs
                                                            sec                               * Used with SSO
                                                            work                             * Logs instance operational files
                                                SYS  ->
                                                            exe  ->
                                                                        run                   * SAP Instance Executables
                                                            gen
                                                            global
                                                            profile                          * SAP Instance Profiles
                                                            sec                  
                                                trans  ->
                                                            EPS  ->
                                                                        in                     * Storage for support packages
                                                                        out
                                                            bin                               * Store common TMS configuration
                                                            buffer                           * Stores the instance TMS queues
                                                            cofiles                          * Header files for transports
                                                            data                             * Data files for transports
                                                            etc                              
                                                            log                               * Logs TMS operational files
                                                            sapnames                    * User names of users using TMS
                                                            tmp                              * TMS Temporary storage

So from this chart, we can see that the TMS files are stored in /usr/sap/trans, the instance profiles in /usr/sap/<SID>/SYS/profile, our instance logs and work file in /usr/sap/<SID>/DVEBMGS00/work, and our SAP executables in \usr\sap\<SID>\SYS\exe\run.  If we had a DI instance attached to this instance, there would be a /usr/sap/<SID>/D01 directory as well.  DEV’s profile directory would be called /usr/sap/DEV/SYS/profile… and on and on and on.  This is universal regardless of the operating system used – the “/” just become “\”.

When a SAP installation runs successfully, two aliases are created – sapmnt and sapalloc.  They are both point to the same thing - \usr\sap.  So you could also access the trans directory by using /sapmnt/trans.

It is important to remember that SID and System Number are two different things.  One is the System Identifier – the three character label for the instance – and one is the System Number – more or less the instances order of installation on the server.  Most instance System Numbers will be 00 but more and more products are coming out to interfere with this numbering scheme.  For example, if during post-installation work you install the Java Add-on, this will create an instance 01 (or whatever the highest system number is + 1) although it is not a ABAP SAP instance, it is a Java SAP instance.  If you go to install a second SAP instance of this same server, the System Number for this new instance would default to 02.  You can manually override this if you wish, and can use any system number up to 98 as long as it is not already in use.  Don’t use 99 either since SAP uses that one for OSS connection.
                                               

XXI.   Where on the OS level can I look for help with SAP related problems?

Your /usr/sap/<SID>/DVEBMGS00/work directory can sometimes be your best friend.  If anything bad happens when you start your SAP instance, it will be recorded here, usually in the dev_rfcx, dev_ms, dev_wx, and stderrx files.  Those x’s stand for numbers because several of them can be created at any given time.   Anything with the word *old* in it is from the previous Sap startup and you can ignore those files.

If a SAP instance does not come up, go to the work directory, sort the files descending by date/timestamp with “ls -ltr” for descending or “ls –lt” for ascending, and start opening files to searching for problems.  99 times out of 100 you will find more information regarding the problem                                           

XXII.   And if I don’t find my problem in the work directory or don’t understand what the error message is telling me?

SAP supplies tools to help you help yourself.  Write down or cut-and-paste (if your message is from a SAPGui screen, using CTRL+Y and CTRL+C can do this) any error messages, and then go to http://service.sap.com/notes.  You have to use that “S” number we talked about earlier to log on here.  Once you are in, you will see SAP Notes – aka OSS Notes - search screen that allows you to paste or key search criteria.  Start broad and narrow your search scope until you think you might have an idea as to your problem.  But make sure that the SAP release for the note matches your SAP release or the note is probably worthless to you.

Can’t find anything in SAP Notes to help you?  You can go to http://service.sap.com/message and open a problem directly with SAP.  A few warnings about doing this – don’t set the priority of your message to Highest unless this is a PRD instance that is down.  SAP gets cranky about that – I would leave the priority at Medium unless you are dealing with a real show stopper.  Also, expect to wait a day or two for a non-PRD problem.  And make sure that you have a working OSS connection so that SAP can log into your SAP landscape – more about this in the next section.  Be sure to provide other information like OS vendor, release and SP level, database vendor, release and SP, kernel patch level, support package levels, etc. so that SAP doesn’t have to ask or dig the information out.

You might get faster results by tapping the knowledge of your fellow SAPer’s on one of the more popular SAP websites.  SAP Fans, SAP Genie, and SAP Super Users have forums where you can post your problem, and SAP Super Users contains Knowledge Repositories where you can pilfer hints, tricks, and techniques without going through the pain of figuring something complex out yourself.  And turnaround can be within minutes if you get lucky and someone out there is really bored or an insomiac.


XXIII.   A working OSS whatie?

As you have probably figured out by now, the technical part of the SAP company was once called OSS or SAPNet.  Then it was changed to SAP Frontend Support.  And now all this is folded into the SAP Marketplace.  Here is where you can download software and patches, add new “S” numbers, get your permanent license keys, register ABAP developers, send messages to SAP, and just about anything SAP related you will ever need to do.  The old way of communication to SAP was using transaction OSS1 which went out, knocked on SAP’s door, and allowed you in with the proper user ID and password.  This was called an OSS – stands for Online SAP Support or something close.  OSS1 will no longer be in use as of April, 2006, and all its functionality has been replaced by the SAP Marketplace at http://service.sap.com.  But you still see tons of OSS references within SAP Notes so just get use to dealing with multiple terms for the same thing which is a norm for SAP.

In the past, a new SAP client was looking at installing a X.25 or Frame Relay connection in order to have a direct connect to the SAP servers, and you normally had to go through a 3rd party to get this work and its configuration done.  Now, it is a simple matter of obtaining a public IP address, setting up a server in your DMZ, installing a SAP product called saprouter, and setting the permissions table.  However, this seems to be one of the implementation tasks that every client keeps putting off until there is a major problem on some sort, and SAP needs to get into your landscape, and you have no OSS connection up and running.  So the importance of getting your Remote Connection Data Sheet completed and FAXed to SAP cannot be over stated.  It really isn’t hard to fill out, and it should only take about half an hour at most.  In return, SAP will FAX you the information you need to complete your setup to one of the SAP servers – there are different ones supporting different OSS configurations and different parts of the world.  

Get it done early and keep a monkey the size of King Kong off your back when you hit a problem.


XXIV.   Why do we need a Solution Manager instance?  What does it do?

Once you get your OSS link to SAP up and running, SAP support can be used via transaction OSS1 until April 2006.  At that point, all SAP support activates must be done either via the SAP Marketplace or via new functionality of your Solution Manager instance. 

Think of a Solution Manager as the hub of your SAP landscape.  It contains information about your SAP instances – called satellite systems.  This information includes the instance patch levels, clients, add-ons, etc.  It does this via Remote Function Calls (RFC) we talked about earlier from the Solution Manager instance to the satellite instances.  And SolMan gathers this information on a scheduled basis to keep it fresh.

As of April when SAP needs to get into one of your instances, you will open a connection for them in the SolMan instance, and they will use SolMan to connect to the instance with the problem.  Instead of having to write down all the pertinent information about the instance having problems, all the information SAP needs like patch level, kernel version, etc. will be easily displayed by SAP by the SolMan instance.  Your OSS connection will be added and configured via SolMan. 

Opening a service connection will use a different procedure than the previous method as well.  First, the LOP program needs to be run on the box hosting saprouter, and a Service Connector Setup Program needs to be run on a workstation holding a SAPGui so that OSS1 can sorta still work the way it used to, and a whole lot of other things we haven’t done yet and have to research in order to explain it to you.  Just keep in mind, the whole process is going through a major change and when we understand, we will pass the knowledge on.

Solution Manager does a whole slew of other functions as well including customer service and help desk tasks.  Since you have to have it by April 2006, you may as well include it as part of your SAP landscape now.