- Регистрация
- 3 Окт 2019
- Сообщения
- 11
- Реакции
- 24
PERFECT WORLD
Installation Guide
IT Operations Manual
Version: 1.0
Date: February 12, 2007
Hardware and Software Requirements
These are the hardware and software requirements of Perfect World to run the game.
I. Game Server Group
Game Manager
Hardware Requirements:
Software Requirements:
Hardware Requirements:
Software Requirements: Network Boot
Database and Backup
Hardware Requirements:
Software Requirements: Network Boot
Spare Server
Hardware Requirements:
II. AU Server Group
AU Manager
Hardware Requirements:
AUTH
Hardware Requirements:
AUDB
Hardware Requirements:
Patch Server 1, Path Server 2 and Patch Server 3
Hardware Requirements:
Patch Server 4
Hardware Requirements:
IV. Log Server
Hardware Requirements:
Hardware Requirements:
Hardware Requirements:
Architecture
I. Server Definitions
Game Server Group
The Game Server Group handles all the game related processing and data. It has 8 servers – game manager, delivery, 3 game servers, 1 backup game server, database, backup and link.
AU Server Group
This group contains the information on account and billing. It also contains the java classes that will be used by the web services.
Patch Server Group
This group handles all the patches in different versions.
Log Server
This server contains all the logs that are being generated by the Game. From the Game Server, the log gets downloaded to this server and processed for migration to the database. This data is being used by the Log Application.
GM Server
This server hosts the GM Tool that is being used by the Customer Support and IT for Game Administration.
Test Server Group
This server group is a replica of the Game Server group only that this group is mainly for testing of Game updates.
Spare Server
This server can be used to substitute for any server in any of the groups.
II. One Server Group Setup
This is the proposed architecture for Perfect World setup.
III. Two Server Group Setup
This is the proposed architecture for a two-server group setup.
This is basically the same as the One-Server group only that there is another group of Game Server Group (Game Server Group 2).
Perfect World Package
The main installation package of the Perfect World is cnet.feilvbin.tar.gz. It contains the following files:
configfeilvbin (folder)
Other packages:
nationalgm.rar – package of the GM Server Tool
Installation and Configuration
I. Game Server Group
To setup the Game Server Group, do the following:
Configuration of Manager Server
Before starting with the configuration of the Game Manager Server, it is assumed that the dump file is already in a laptop and is connected to the internal network where the Manager server is.
To configure the Manager Server, the user should do the following subtasks:
Prepare package
RAID Configuration
DRAC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
Prepare the package
Before going into the details of the configuration, do the following first:
RAID Configuration
Press CTRL+R to load the RAID BIOS configuration interface.
Select Controller 0. Press F2 then select ‘Reset Config’ to delete the existing configuration.
The interface will display ‘No configuration present!’. Press F2 and select ‘Create New VD’ to create a new configuration.
The Virtual Disk Management screen will be loaded and do the following:
On the RAID Level field, choose ‘RAID-1’ and Press ‘Enter’.
Cross out the squares corresponding to the two drives in this partition by pressing space bar. The first drive is the root, second drive is the swap and the third is export.
First, set the Basic Settings for the Root.
Root:
VD Size: 10000 MB
VD Name: Root
Cross out the ‘Advanced Setting’ and set the following values for the partition.
Element Size: 64 MB
Read Policy: Adaptive Read
Write Policy: Write Back
Cross out Initialize by pressing spacebar.
Create the next two partition by going to ‘Space Allocation’ under free space and press F2. Repeat steps iii to v. Use the following values for the corresponding partitions.
Swap:
VD Size: 4096 MB
VD Name: Swap
Export:
VD Size: whatever the remaining space is
VD Name: Export
Press Esc and click OK to save.
Press CTRL+ALT+DEL to reboot the server.
DRAC Configuration
NIC Selection – Dedicated
LAN Parameters
Ethernet IP Address (specify the internal IP; refer to the IP assignments in the Appendix)
Default Gateway (specify the internal IP of the gateway)
Host Name String (input ‘Manager’)
LAN User Configuration
Enter Username: root
Enter Password: <input password here>
Confirm Password: <input password here>
Press Escape and Save Changes and Exit. The server will reboot.
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers. These will be needed later in the modification of the XML.
BIOS Configuration
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
Go to ‘Integrated Devices’ and do the following:
Set the value of the Embedded NIC1 enabled with PXE to true.
Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller set’ to Off.
Go to ‘Serial Communication’ set the value to On without Console Redirection.
Go to ‘External Serial Connector’ set to Remote Access Device.
Go to ‘System Security’ set to AC Power Recovery On.
Go to ‘Report Keyboard Errors’ set to Do not Report.
Save the Changes and Exit.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and follow the procedures below to install
Before creating the partitions, check if there are any existing partitions. To do this type this command cat /proc/partitions. This will display the virtual directories and the partitions for each directory. The mapping of the virtual directories are as follows:
sda – root
sdb – swap
sdc – export
If there are no partitions, proceed to the next step.
Run fdisk /dev/sda to create partition on the first virtual directory.
Press p to check the original partition of the directory.
Press n to create a new partition.
Press p to create a primary partition. Press 1 to use the default partition value.
Use default value and press Enter twice. Press p again to view created partition. This should display the device boot /dev/sda1.
Press w to save the changes in the partition table.
Format swap by executing this command – mkswap /dev/sdb.
Format sda and sdc by executing these commands:
mkfs.xfs /dev/sda1 –f
mkfs.xfs /dev/sdc -f
Go to cd /tmp to create a new directory for sda and sdc. Issue the following commands:
cd /tmp
mkdir sda1 sdc
Mount the directories created by executing the following commands:
mount /dev/sda1 /tmp/sda1
mount /dev/sdc /tmp/sdc
Dump folder contains the archived files for billing and the games. The billing-related archives are the files with this filename – OpenBeta_AU_ *. The game-related archives are the ones with filename – hw_game_*. Notice that for each archive groups, there are files with export and root on their filenames. The archive with root on the filename consists of files used for system management. The one with export on the filename consists of files that are used for the system that manages all the other non-disk servers.
Copying of files may create inconsistencies between the files, to check this issue the following commands:
cat hw_game_root_dump.gz.md5
md5sum hw_game_root_dump.gz
This step is optional and is used only to check the files.
Extract the archives to the corresponding directories by issuing the following commands:
zcat hw_game_root_dump.gz | xfsrestore - /tmp/sda1
zcat hw_game_export_dump.gz | xfsrestore - /tmp/sdc
This operation should return this message: RESTORE STATUS: SUCCESS. If this is not the return message, redo the process from the creation of the partitions.
Configuration of the Boot Loader (GRUB)
After the extraction of the image file, configure the boot loader. To do this, follow the commands below:
To check if the kernel installed is the one that’s needed, do the following commands before rebooting the system:
cd /tmp/sda1/boot/grub
cat menu.lst
On this file, the version of the kernel should be equal to 2.6.17.11.management.1950. Again, this step is optional so this can be skipped.
Reconfigure BIOS
After the reboot do the following:
To configure the Backup Server, the user should do the following subtasks:
RAID Configuration
BMC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
RAID Configuration
The procedure for the configuration of the RAID for this server is pretty much the same except that only two virtual directories are created – swap and export. Just the same swap will use 4096 MB space and the rest of the free space is going to be used by the export.
BMC Configuration
The configuration of BMC is basically the same as DRAC since it serves the same purpose only that BMC can not be accessed using an external IP. Change the following selections to the corresponding values:
IPMI over LAN - On
NIC Selection – Shared
LAN Parameters:
Ethernet IP Address (specify the internal address of the BMC which is equal to the x.x.x.132 for Backup and x.x.x.131 for Database)
Host Name String (input ‘Backup’ for Backup and ‘Database’ for Database)
LAN User Configuration:
Username – default to root
Password: <input password>
Confirm Password: <input password>
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers; these will be needed later in the modification of the XML.
BIOS Configuration
Configuration of the BIOS for Backup and Database Server follows the same procedure as the one used in Manager.
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
Save the Changes and Exit.
Edit the XML to allow the Manager to reach the other servers in the network
In the root folder in cnet directory, configfeilvbin folder contains feilvbin.xml, a file being used by the Manager to access all the servers in the Game Server network. This XML file has been customized to fit the requirements of LU! PH. To edit the file, follow the procedures below:
Move the files – package and the XML from cnet to the root directory by following the
procedure listing below:
scp root@VMWareIP:/export/cnet.feilvbin.tar.gz /root
Supply the the root password
ifconfig eth0 IP
where IP is the IP Address of the Manager.
tar –zxvf cnet.feilvbin.tar.gz – uncompress the cnet.feilvbin folder.
cd cnet.feilvbin
cp cpackage.feilvbin.tar.gz /root – must be in /root/cnet.feilvbin/configfeilvbin directory
cd
tar –zxvf cpackage.feilvbin.tar.gz
mv cpackage.feilvbin cpackage
vi remount.sh
Remove the servers that do not belong to the group. Note that this script can be found at root cpackage/tool.
cd cnet.feilvbin
cp feilvbin.xml /root/cpackage/
vi feilvbin.xml
After reboot, execute the following commands:
cd /etc/sysconfig/network-scripts/
mv ifcfg-ppp0 ofcg-pp0.old
This is used to disable VPN access. For LU! setup, this may be skipped since
Install the package by executing this command:./package feilvbin.xml install.
Reboot the Server.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and create the partitions and format the directories of both Database and Backup:
Since both Backup and Database have only two partitions – swap and export. The mapping of the virtual directory is specified below:
sda – swap
sdb – export
Format the directories using the following commands:
For Database server:
rlogin database
mkswap /dev/sda
mkfs.xfs /dev/sdb
exit
For Backup server:
cp /sbin/mkfs.xfs /export/backup/sbin/
rlogin backup
mkswap /dev/sda
mkfs.xfs /dev/sdb
exit
Check if the partition exist, type the command “cat /proc/partitions” (view the partitions in the server)
Reboot the server.
Edit the Non-Disk Servers
For non-disk servers, only the BMC and the BIOS gets configured during setup. Refer to the configuration setup for BMC and BIOS for Database and Backup. After the configuration, reboot the servers from the Manager window by issuing the command:
This command will restart the server. To check if the server is already up, do the following:
ping game 1 2 3, link, delivery,database, backup
login into the server by typing rlogin game1
type ifconfig
type ethtool eth0
Publish the Game
After setting up the server, the game should now be installed. To install the game, issue this command on the Game Manager Server:
cd cpackage
./package feilvbin.xml gameinstall.
Reboot the computer after the installation is complete.
Provide Access to the Game Server Setup
To provide client PCs access to the Game Manager, the certificates should be installed in these computers. Certificates have been provided by PW (refer to the cnet.doc in Appendix B). This access is through the web using this URL:
To be able to activate this page on the Game server, add the settings to the environment variables in the profile (.bash profile). Do the following:
vi .bash_profile
PATH=$PATH:$HOME/bin:/usr/local/jakarta-tomcat-5.5.9/bin
Export JAVA_HOME=/usr/java/jdk1.5.0_05
Log out to reload the system and use these variables.
Remove the old java program and install the new java provided by PW.
cd /usr/
mv java java.bak
cd /root/cnet.feilvbin/
cp java.tar.gz /usr
cd /usr
tar zxvf java.tar.gz
Run the Tomcat Java program:
cd /usr/local/jakarta-tomcat-5.5.9/bin
./startup.sh
Note: The latest .bash_profile given by PW already contains this modification so this step can be skipped.
II. AU Server Group
AU Server group contains the account and billing data as well as the web services that will be used for the game. To set up this server group, do the following:
AU Manager is the server responsible in managing the Account and Billing DB as well as the web server. To configure the manager do the following:
RAID Configuration
DRAC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
RAID Configuration
Press CTRL+R to load the RAID BIOS configuration interface.
Select Controller 0. Press F2 then select ‘Reset Config’ to delete the existing configuration.
The interface will display ‘No configuration present!’. Press F2 and select ‘Create New VD’ to create a new configuration.
The Virtual Disk Management screen will be loaded and do the following:
On the RAID Level field, choose ‘RAID-1’ and Press ‘Enter’.
Cross out the squares corresponding to the two drives in this partition by pressing space bar. The first drive is the root, second drive is the swap and the third is export.
First, set the Basic Settings for the Root.
Root:
VD Size: 10000 MB
VD Name: root
Cross out the ‘Advanced Setting’ and set the following values for the partition.
Element Size: 64 MB
Read Policy: Adaptive Read
Write Policy: Write Back
Cross out Initialize by pressing spacebar.
Create the next two partitions by going to ‘Space Allocation’ under free space and press F2. Repeat steps iii to v. Use the following values for the corresponding partitions.
Swap:
VD Size: 4096 MB
VD Name: swap
Export:
VD Size: whatever the remaining space is
VD Name: export
Press Esc and click OK to save.
Press CTRL+ALT+DEL to reboot the server.
DRAC Configuration
Go to the Remote Access Configuration by pressing CTRL+E after reboot.
Change the values of the following:
IPMI Over Lan – On
NIC Selection – Dedicated
LAN Parameters
Ethernet IP Address (specify the internal IP the IDC)
Default Gateway (specify the internal IP of the gateway)
Host Name String (input ‘Manager’)
LAN User Configuration
Enter Username: root
Enter Password: <input password here>
Confirm Password: <input password here>
Press Escape and Save Changes and Exit. The server will reboot.
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers. These will be needed later in the modification of the XML.
BIOS Configuration
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
Go to ‘Integrated Devices’ and do the following:
Set the value of the ‘Embedded NIC1 enabled with PXE’ to true.
Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller’ set to Off.
Go to ‘Serial Communication’ set the value to On without Console Redirection.
Go to ‘External Serial Connector’ set to Remote Access Device.
Go to ‘System Security’ set to AC Power Recovery On.
Go to ‘Report Keyboard Errors’ set to Do not Report.
Save the Changes and Exit.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and follow the procedures below to install:
Create partitions and Format Directories.
Before creating the partitions, check if there are any existing partitions. To do this type this command cat /proc/partitions. This will display the virtual directories and the partitions for each directory. The mapping of the virtual directories are as follows:
sda – root
sdb – swap
sdc – export
If there are no partitions, proceed to the next step.
Run fdisk /dev/sda to create partition on the first virtual directory.
Press p to check the original partition of the directory.
Press n to create a new partition.
Press p to create a primary partition. Press 1 to use the default partition value.
Press p again to view created partition. This should display the device boot /dev/sda1.
Press w to save the changes in the partition table.
Format swap by executing this command – mkswap /dev/sdb.
Format sda and sdc by executing these commands:
mkfs.xfs /dev/sda1 –f
mkfs.xfs /dev/sdc -f
Mount the Partitions
Go to cd /tmp to create a new directory for sda and sdc. Issue the following commands:
cd /tmp
mkdir sda1 sdc
Mount the directories created by executing the following commands:
mount /dev/sda1 /tmp/sda1
mount /dev/sdc /tmp/sdc
Extract Image to the Manager’s Disk
Go to the directory where the image is located. Run this command – cd /dump.
Dump folder contains the archived files for billing and the games. The billing-related archives are the files with this filename – OpenBeta_AU_ *. The game-related archives are the ones with filename – hw_game_*. Notice that for each archive groups, there are files with export and root on their filenames. The archive with root on the filename consists of files used for system management. The one with export on the filename consists of files that are used for the system that manages all the other non-disk servers.
Copying of files may create inconsistencies between the files to check this issue the following commands:
cat Release_AU_root_JP.dump.gz.md5
md5sum Release_AU_root_JP.dump.gz
This step is optional and is used only to check the files.
Extract the archives to the corresponding directories by issuing the following commands:
zcat Release_AU_root_JP.dump.gz | xfsrestore - /tmp/sda1
zcat OpenBeta_AU_export.JP.dump.gz | xfsrestore - /tmp/sdc
This operation should return this message: RESTORE STATUS: SUCCESS. If this is not the return message, redo the process from the creation of the partitions.
Configuration of the Boot Loader (GRUB)
After the extraction of the image file, configure the boot loader. To do this, follow the commands below:
To check if the kernel installed is the one that’s needed, do the following commands before rebooting the system:
cd /tmp/sda1/boot/grub
cat menu.lst
To avoid kernel panic after the aumanger is rebooted, edit the grub file default value to 0. From /root directory type the command:
vi /tmp/sda1/boot/grub/menu.lst
default=0
On this file, the version of the kernel should be equal to 2.6.17.11.management.1950.
Reconfigure BIOS
After the reboot do the following:
To configure AUTH, do the following procedures:
Configure BIOS
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
The configuration of BMC is basically the same as DRAC since it serves the same purpose only that BMC can not be accessed using an external IP. Change the following selections to the corresponding values:
NIC Selection – Shared
LAN Parameters:
Ethernet IP Address (check the IP Address assignments in the Appendix section of this IP)
Host Name String (input ‘AUTH’)
LAN User Configuration:
Username – default to root
Password: <input password>
Confirm Password: <input password>
Install Package via Network (Aumanager)
To install the package via network, run the following commands:
scp root@laptopid:/export/cnet.feilvbin.tar.gz /root
tar –zxvf cnet.feilvbin.tar.gz
Update configuration and install the package.
Edit aumanager.xml. Change auth kernel to vmlinuz-2.6.17.11.nodisk1950,
After editing, copy the kernel to pxeboot by executing the following commands:
cd /boot
cp vmlinuz-2.6.17.11.nodisk.1950 /tftpboot/linux/pxeboot
cd /root/cnet.feilvbin/
cp aumanager.xml /root/package_only_for_au/config/
cd /root/package_only_for_au
Install the package by running this command:
./package config/aumanager.xml install
reboot
Configure java in the AUTH and AU Manager
Configure java in AU Manager:
Log in to AU Manager Server as root.
Go to the cnet directory – cd /root/cnet.feilvbin
Remove the current installation of java.
rm –rf /usr/java
.
Copy the archived java setup files to the usr directory – cp java.tar.gz /usr/.
Go to the usr folder – cd /usr/.
Extract the archive to this directory by executing this command - tar zxvf java.tar.gz. This will install the jdk to the server. The jdk version that is being used for Perfect World is 1.5.0_05.
This will extract the jdk to the server.
Go to the library of java directory – cd java/jdk 1.5.0_05/lib/.
Copy the bash profile from the cnet to the server, Run these commands:
cd /root/cnet.feilvbin
cp bash_profile /root/.bash_profile
ipmish –ip authBMC IP –u “root” –p “password” power cycle
Configure java in AUTH:
Go to the cnet directory – cd /root/cnet.feilvbin
Remove the current java installation.
rm –rf /export/auth/usr/java
Copy the archived java setup files to the usr directory of auth – cp java.tar.gz /export/auth/usr/.
Go to the usr folder – cd /export/auth/usr/.
Extract the archive to this directory by executing this command - tar zxvf java.tar.gz. This will install the jdk to the server. The jdk version that is being used for Perfect World is 1.5.0_05. This will extract the jdk to the server.
Copy the files from cnet to the server. Run the following commands:.
cd /root/cnet.feilvbin
cp bash_profile /export/auth/root/.bash_profile
Configure the SSL
Remove any existing SSL configuration: rm –rf /etc/ssl.
Copy the SSL archive from cnet.feilvbin to etc – cp ssl.tar.gz /etc/
Go to etc folder and extract the archive.
cd /etc/
tar zxvf ssl.tar.gz
Configure Hosts
Edit the host file aumanager – vi /etc/hosts. Entries in the file should look like this:
x.x.x.254 aumanager
x.x.x.1 auth
x.x.x.10 audb
The IP should be the internal address of the servers. Save and Exit.
Edit the host file in auth – vi /export/auth/etc/hosts/. Entries in the file should look like this:
x.x.x.254 manager
x.x.x.1 auth
x.x.x.10 audb
Login to auth and check the configuration. Run the following commands:
rlogin auth
ifconfig
Set Up Account and Billing DB (audb)
vi table.xml
cd /root/cnet.feilvbin/configfeilvbin
cp table.xml /etc/
cp table.xml /export/auth/etc/
vi gmopen
cp gmopen.xml /etc/
cp gmopen.xml /export/auth/etc/
cd /root/cent.feilvbin/configfielvbin
cp authd.conf /export/auth/etc/
No need to edit this file, Perfect World will provide a customized authd.conf file to cater to the needs of LU! Philippines.
vi GMServer.conf
Save and Exit.
cp GMServer.conf /etc/
cp cauthd.tar.gz /export/auth/root/
cd /export/auth/root/
tar zxvf cauthd.tar.gz
This creates a folder cauthd in the auth server.
./authd & is used to boot AUTH. The result of running this command should be authd: add PollIO task. When this is the output, it means that the boot is successful.
Configure tomcat
Go to the webapps directory in aumanager and clear the contents of the folder. Execute the following commands:
Issue the following commands to start tomcat.
cd /usr/local/Jakarta-tomcat-5.5.9/bin
startup.sh
To check if the tomcat is up, access the webmanager interface – https://x.x.x.254/monitor/ where x.x.x.254 if the internal IP of the aumanager.
Configure the Account and Billing Database
The Account and Billing DB uses Windows 2003 for OS and MS SQL Server 2000 Enterprise Edition.
Open the .sql files (mssql_table.sql and mssql_storedprocedure.sql) and run in the Query Analyser to build the database. Mssql_table.sql creates the database and the tables while mssql_storedprocedure.sql creates the stored procedure being used by the system. Remember delete all the comments before running the commands in the Query Analyzer.
Once everything has been created, create user to access the database.
account: fielvbin
db: audb
permission: dbowner
Reconfigure AU Manager and AUTH Servers
Go to AU Manager and edit table.xml.
cd /etc/
vi table.xml
Change the IP of the database, database name, username and password. Save and exit.
Go to AUTH and edit table.xml
vi /export/auth/etc/table.xml
Change the IP of the database, database name, username and password. Save and exit.
Reboot AUTH – ipmish -ip 192.168.1.100 -u “root” -p “password” power cycle.
After reboot, run authd on AUTH Server and exit. Execute the following commands:
Go back to the AU Manager and restart tomcat. Execute the following commands:
Check the connection of AUTH and the Account and Billing Database
To check the connection between AUTH and the Account and Billing Database, access the webmanager system.
Access http://aumanagerip/monitor via web. This is the tool that checks the connection of the aumanager and the audb.
A popup screen will be displayed, choose iwebmanage. Accept the certificate to enter the site.
Try to register a new user. Verify by checking the existence of the record on the database.
Note: If there are any errors, check the logs of the server using this command: tail -f /usr/local/jakarta-tomcat-5.5.9/logs/catalina.out.
III. Patch Server Group
Perfect World did not provide an installation package for the patch server, installation will be done remotely by PW. Below is a summary of the work that needs to be done by Level Up! and Perfect World to set up this server group:
Level Up! will:
prepare the servers, install necessary software – OS (Fedora Core2), apache, FTP
install firewall in this setup
open these servers to Perfect World so they will have access.
If one of the servers goes down, LU! just has to reinstall all the necessary software and seek PW for support.
Perfect World will:
configure servers and install the clients that will be used for the upload and download of the patch.
In case one of the patch servers goes down, PW will provide support and reconfigure the patch server.
IV. GM Server
GM Server contains the GM Tool, the web application used by Customer Support. The web application and the database can be hosted in separate servers but in this document it is assumed that they will be hosted on the same server. The GM Server communicates with AU and iweb as illustrated below:
To access the tool the user must have a certificate stored in USB drive. This certificate is created and issued to users who need to access the site.
To access the tool:
For LUPH implementation, the certificate will not be used. So the for the access of this server, straight forward username and password validation through the database will be used.
To setup the server, do the following:
<property name=”hibernate.connection.url”> <!CDATA[jdbc:mysql://gmserverIP:3306/csm?useUnicode=true&characterEncoding=GBK&autoReconnect=true]]>
</property>
<property name=”hibernate.connection.username”>dbusername</property>
<property name=”hibernate.connection.password”>dbpassword</property>
where:
gmserverIP – pertains to the IP of the server where the GM Tool is deployed.
dbusername and dbpassword – are the username and password used by mysql.
webservice
url = https://aumanagerIP:443/GMServer/gmservice
userurl = https://aumanagerIP:443/GMServer/userservice
where:
aumanagerIP – pertains to the IP of the au manager server
Copy the file to WebModule\WEB-INF\classes\.
Rename the file to csmInfo.properties.
V. Log Server
The logs are generated in the Game Server (/var/log). These logs are as follows:
brief.tar.gz
contains the character and faction information
downloaded from the Manager(game) to the Log Server
archive contains CSV files which are dumped into the database
tables that get populated with this data are: chardata, faction, factionuser
logs can be viewed using the web application for the logs – http://logserverIP/logs/defaul.htm
retention period: 30 days
world2.formatlog.1.gz
contains the game details of the accounts.
downloaded from the Manager(game) to the Log Server.
archive contains CSV files which are dumped into the database
sample tables that gets filled up by this data are: AccountLog, RemoteLog, UpgradeLog
logs can be viewed using the web application for the logs – http://logserverIP/logs/default.htm
retention period: 30 days
world2.err.8
contains the server error logs
not downloaded to the Log Server
user needs to read through the log to check the details because this can not be viewed through web.
retention period: 15-30 days on the Game Manager.
world2.chat.1.gz
contains the chat logs
not downloaded to the Log Server
can be queried using a web interface – http://logserverIP/kchat/index.cgi
retention period: 15-30 days on the Game Manager.
world2.log
contains the character details
can be queried using a web interface – http://logserverIP/kcfgi/index.cgi
Among these logs, only brief.tar.gz and world2.formatlog.1.gz are downloaded to the Log Server. The download happens at 4:00 AM. These logs are analyzed and dumped into the database in the Log Server at 5:05 AM. Note that if there is more than one server group, the download of the backup maybe done simultaneously. But it should be ensured that the download of the logs finish before the job to dump these logs into the database starts.
Log Server has no installation package. For the setup the following items are the summary of the work that needs to be done by Level Up! and Perfect World:
Level Up! will:
prepare the server, install necessary software – Fedora 4
create the following partitions
File System capacity
/dev/sda2 40G
/dev/sda1 99M
tmpfs 1011M
/dev/sda5 236G
allow and give the IP of the server to Perfect World for remote access
Perfect World will:
deploy the Log Tool on the server
provide remote support
Operations
Restarting the System
When restarting the servers in the network, the proper sequence to be followed is:
Monitoring is done using cricket in iweb. Cricket is a system used to monitor trends in time-series data. It helps the network managers visualize and understand the traffic on their networks. To access this tool, go to https://gamemanagerIP/iweb/monitor and click on the Monitoring Information link. The tools available for this link are as follows:
Cricket
Syslog
Route (trace route)
Views (monitoring graphs)
The graphs displayed on this page are as follows:
Maintenance
Game Server Maintenance is done primarily by using the web management tool.
Installation Guide
IT Operations Manual
Version: 1.0
Date: February 12, 2007
Hardware and Software Requirements
These are the hardware and software requirements of Perfect World to run the game.
I. Game Server Group
Game Manager
Hardware Requirements:
- DELL 1950
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- 2x146Gig Hard Disk 10K SAS
- PERC 5/i: SAS 3.0 Gb/s RAID controller with 256MB of battery-backup cache
- DRAC
- Dual Power Supply
Software Requirements:
- Perfect World Package (deployed via network; Perfect World proprietary)
Hardware Requirements:
- DELL 1950 (Diskless)
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- Single Power Supply
Software Requirements: Network Boot
Database and Backup
Hardware Requirements:
- DELL 1950
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- 2x146Gig Hard Disk 10K SAS
- PERC 5/i: SAS 3.0 Gb/s RAID controller with 256MB of battery-backup cache
- Single Power Supply
Software Requirements: Network Boot
Spare Server
Hardware Requirements:
- DELL 1950 (Diskless)
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- 2x146Gig Hard Disk 10K SAS
- PERC 5/i: SAS 3.0 Gb/s RAID controller with 256MB of battery-backup cache
- DRAC
- Dual Power Supply
II. AU Server Group
AU Manager
Hardware Requirements:
- DELL 1950
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- 2x146Gig Hard Disk 10K SAS
- PERC 5/i: SAS 3.0 Gb/s RAID controller with 256MB of battery-backup cache
- DRAC 5
- Dual Power Supply
AUTH
Hardware Requirements:
- DELL 1950 (Diskless)
- 2xDual Core 1.6Ghz/4MB/1066Mghz/ Intel Xeon5110 Processors
- 2GB Memory 533Mghz DDR -2 ECC
- Single Power Supply
AUDB
Hardware Requirements:
- HP DL 380
- 2x3.0Ghz Intel Xeon
- 2GB Memory
- 2x36GB 15K USCSI
- 4x146GB 15K USCSI
- RAID Controller
- Dual Power Supply
- Windows 2003 Std R2
- MS SQL 2000 Ent.
Patch Server 1, Path Server 2 and Patch Server 3
Hardware Requirements:
- IBM x335
- 2x2.4 GHz CPU
- 2 GB RAM
- 2x36GB HDD
- Integrated RAID Controller
- Dual Power Supply
Patch Server 4
Hardware Requirements:
- HP DL320
- 2 x 3 GHz with Hyper Threading CPU
- 2x73 GB RAID1
- 2 GB RAM
- Integrated RAID Controller
- Dual Power Supply
IV. Log Server
Hardware Requirements:
- DL 380
- 2x3.0Ghz Intel Xeon
- 2GB Memory
- 6x146GB HDD (RAID 5)
- Integrated RAID Controller
- Dual Power Supply
- Fedora Core 4 (use bundled php and mysql)
- Proprietary package (to be installed remotely by Perfect World)
Hardware Requirements:
- IBM x335
- 2x2.4 GHz CPU
- 2GB Memory
- 3x36 GB HDD
- Integrated RAID Controller
- Dual Power Supply
- Propriety package (deployed via network; from AUManager dump)
Hardware Requirements:
- Switch (Production) – Cisco 3560 24T (External & Internal)
- Switch (Test) – Cisco 2950 (External & Internal)
- Firewall – FG 3000
- OB License (AUDB) – Windows 2003 Std R2
- DB License (AUDB) – MS SQL 2000 Ent.
Architecture
I. Server Definitions
Game Server Group
The Game Server Group handles all the game related processing and data. It has 8 servers – game manager, delivery, 3 game servers, 1 backup game server, database, backup and link.
- Game Manager – this server contains the OS of all the non-disk servers which manages the operations of all these servers. It also acts as a DHCP server in the internal network. Also acts as TFP and NFS server.
- Delivery Server – handles the communications of the server within the network. This server has a public and private IP and has no disk.
- Game Servers – these servers will contain the maps of the game.
- Game Database – contains the character information and other game related data. This server has a hard disk.
- Backup – contains the backup of the Game Database. This server is also the one responsible for checking the unique game character checking in the game. When this server is down, the users will not be able to create a new character.
- Link – this is the entry point of the players into the Game Network. It acts as an NAT and has a public and private IP.
AU Server Group
This group contains the information on account and billing. It also contains the java classes that will be used by the web services.
- AU Manager – this server contains the OS of all the non-disk servers in this server group which manages the operations of all these servers.
- AUTH – entry point to the AU Server Group; It is used for login and to authenticate accounts during account creation
- AUDB – contains the account and billing database.
Patch Server Group
This group handles all the patches in different versions.
- Patch 1 – this server where the Chinese version of the patch is uploaded by PW. LU! should use the Chinese client to download the update. LU! will have this patch translated to English. FTP will be used for upload of data and server will be accessed using the domain name.
- Patch 2 – this is where the translated version of the patch is uploaded by LU!. PW will download the translated patch from this server.
- Patch 3 – this server is the client patch server where the final release of the patch is uploaded after testing. This server is open to the public because this will be accessed by the players.
- Patch 4 - this server is the test patch server. The translated patch downloaded from Patch Server 2 will be repackaged by PW and uploaded in this server. LU! will test the patch on this server and if testing is ok, the patch is moved to Patch Server 3.
Log Server
This server contains all the logs that are being generated by the Game. From the Game Server, the log gets downloaded to this server and processed for migration to the database. This data is being used by the Log Application.
GM Server
This server hosts the GM Tool that is being used by the Customer Support and IT for Game Administration.
Test Server Group
This server group is a replica of the Game Server group only that this group is mainly for testing of Game updates.
Spare Server
This server can be used to substitute for any server in any of the groups.
II. One Server Group Setup
This is the proposed architecture for Perfect World setup.
III. Two Server Group Setup
This is the proposed architecture for a two-server group setup.
This is basically the same as the One-Server group only that there is another group of Game Server Group (Game Server Group 2).
Perfect World Package
The main installation package of the Perfect World is cnet.feilvbin.tar.gz. It contains the following files:
- mssql_storedprocedure.sql – stored procedure used to create the SP for account/billing db
- mssql_table,sql – stored procedure used to create tables for account and billing DB
- bash_profile – configuration file for the AU servers
- feilvbin.xml – the configuration file of the whole setup (game server and au server groups only)
- java.tar.gz – archive used to install java in the servers
- ssl.tar.gz – archive used to install the certificates that will be used for access validation
- jakarta-tomcat-5.5.9.aumanager.tar.gz – archive used to install tomcat in the servers
- aumanager.xml – configuration file for au setup
configfeilvbin (folder)
- cpackage.feilvbin.tar.gz – archive file used to install the games
- GMServer.conf – contains the configuration for the GM server.
- gmopen.xml – contains the server list
- table.xml – configuration file of the Database server.
Other packages:
nationalgm.rar – package of the GM Server Tool
Installation and Configuration
I. Game Server Group
To setup the Game Server Group, do the following:
- Configuration of Manager Server
- Configuration of the Backup and Database Servers
- Edit the XML (feilvibn.xml)
- Configuration of the Non-disks Servers.
- Publish the Game
- Providing Access to the Game Server Setup
Configuration of Manager Server
Before starting with the configuration of the Game Manager Server, it is assumed that the dump file is already in a laptop and is connected to the internal network where the Manager server is.
To configure the Manager Server, the user should do the following subtasks:
Prepare package
RAID Configuration
DRAC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
Prepare the package
Before going into the details of the configuration, do the following first:
- Copy package to the laptop.
- Set the IP of the laptop (VMWare) to 192.168.0.1.
- Connect the laptop to the same network as the Game Server or connect it directly to the Manager.
RAID Configuration
Press CTRL+R to load the RAID BIOS configuration interface.
Select Controller 0. Press F2 then select ‘Reset Config’ to delete the existing configuration.
The interface will display ‘No configuration present!’. Press F2 and select ‘Create New VD’ to create a new configuration.
The Virtual Disk Management screen will be loaded and do the following:
On the RAID Level field, choose ‘RAID-1’ and Press ‘Enter’.
Cross out the squares corresponding to the two drives in this partition by pressing space bar. The first drive is the root, second drive is the swap and the third is export.
First, set the Basic Settings for the Root.
Root:
VD Size: 10000 MB
VD Name: Root
Cross out the ‘Advanced Setting’ and set the following values for the partition.
Element Size: 64 MB
Read Policy: Adaptive Read
Write Policy: Write Back
Cross out Initialize by pressing spacebar.
Create the next two partition by going to ‘Space Allocation’ under free space and press F2. Repeat steps iii to v. Use the following values for the corresponding partitions.
Swap:
VD Size: 4096 MB
VD Name: Swap
Export:
VD Size: whatever the remaining space is
VD Name: Export
Press Esc and click OK to save.
Press CTRL+ALT+DEL to reboot the server.
DRAC Configuration
- Go to the Remote Access Configuration by pressing CTRL+E after reboot.
- Change the values of the following:
NIC Selection – Dedicated
LAN Parameters
Ethernet IP Address (specify the internal IP; refer to the IP assignments in the Appendix)
Default Gateway (specify the internal IP of the gateway)
Host Name String (input ‘Manager’)
LAN User Configuration
Enter Username: root
Enter Password: <input password here>
Confirm Password: <input password here>
Press Escape and Save Changes and Exit. The server will reboot.
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers. These will be needed later in the modification of the XML.
BIOS Configuration
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
Go to ‘Integrated Devices’ and do the following:
Set the value of the Embedded NIC1 enabled with PXE to true.
Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller set’ to Off.
Go to ‘Serial Communication’ set the value to On without Console Redirection.
Go to ‘External Serial Connector’ set to Remote Access Device.
Go to ‘System Security’ set to AC Power Recovery On.
Go to ‘Report Keyboard Errors’ set to Do not Report.
Save the Changes and Exit.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and follow the procedures below to install
- Create partitions and Format Directories.
Before creating the partitions, check if there are any existing partitions. To do this type this command cat /proc/partitions. This will display the virtual directories and the partitions for each directory. The mapping of the virtual directories are as follows:
sda – root
sdb – swap
sdc – export
If there are no partitions, proceed to the next step.
Run fdisk /dev/sda to create partition on the first virtual directory.
Press p to check the original partition of the directory.
Press n to create a new partition.
Press p to create a primary partition. Press 1 to use the default partition value.
Use default value and press Enter twice. Press p again to view created partition. This should display the device boot /dev/sda1.
Press w to save the changes in the partition table.
Format swap by executing this command – mkswap /dev/sdb.
Format sda and sdc by executing these commands:
mkfs.xfs /dev/sda1 –f
mkfs.xfs /dev/sdc -f
- Mount the partitions
Go to cd /tmp to create a new directory for sda and sdc. Issue the following commands:
cd /tmp
mkdir sda1 sdc
Mount the directories created by executing the following commands:
mount /dev/sda1 /tmp/sda1
mount /dev/sdc /tmp/sdc
- Extract image to the Manager’s hard disk
Dump folder contains the archived files for billing and the games. The billing-related archives are the files with this filename – OpenBeta_AU_ *. The game-related archives are the ones with filename – hw_game_*. Notice that for each archive groups, there are files with export and root on their filenames. The archive with root on the filename consists of files used for system management. The one with export on the filename consists of files that are used for the system that manages all the other non-disk servers.
Copying of files may create inconsistencies between the files, to check this issue the following commands:
cat hw_game_root_dump.gz.md5
md5sum hw_game_root_dump.gz
This step is optional and is used only to check the files.
Extract the archives to the corresponding directories by issuing the following commands:
zcat hw_game_root_dump.gz | xfsrestore - /tmp/sda1
zcat hw_game_export_dump.gz | xfsrestore - /tmp/sdc
This operation should return this message: RESTORE STATUS: SUCCESS. If this is not the return message, redo the process from the creation of the partitions.
Configuration of the Boot Loader (GRUB)
After the extraction of the image file, configure the boot loader. To do this, follow the commands below:
- Type grub to go to the interface of grub.
- Type setup (hd0) (hd0,0), and press Enter
- Type quit.
- Type reboot.
To check if the kernel installed is the one that’s needed, do the following commands before rebooting the system:
cd /tmp/sda1/boot/grub
cat menu.lst
On this file, the version of the kernel should be equal to 2.6.17.11.management.1950. Again, this step is optional so this can be skipped.
Reconfigure BIOS
After the reboot do the following:
- Go to the ‘BIOS Configuration’ and choose Boot Sequence. Uncheck Embedded NIC 1 MBA Version… option and check Hard drive C:.
- Then go to ‘Integrated Devices’, choose the Embeded GB NIC1. Change the value from Enabled with PXE to Enabled without PXE.
- Save Changes and Exit. The server will reboot. The default password of root in this server is 1234.
To configure the Backup Server, the user should do the following subtasks:
RAID Configuration
BMC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
RAID Configuration
The procedure for the configuration of the RAID for this server is pretty much the same except that only two virtual directories are created – swap and export. Just the same swap will use 4096 MB space and the rest of the free space is going to be used by the export.
BMC Configuration
The configuration of BMC is basically the same as DRAC since it serves the same purpose only that BMC can not be accessed using an external IP. Change the following selections to the corresponding values:
IPMI over LAN - On
NIC Selection – Shared
LAN Parameters:
Ethernet IP Address (specify the internal address of the BMC which is equal to the x.x.x.132 for Backup and x.x.x.131 for Database)
Host Name String (input ‘Backup’ for Backup and ‘Database’ for Database)
LAN User Configuration:
Username – default to root
Password: <input password>
Confirm Password: <input password>
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers; these will be needed later in the modification of the XML.
BIOS Configuration
Configuration of the BIOS for Backup and Database Server follows the same procedure as the one used in Manager.
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
- Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
- Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
- Go to ‘Integrated Devices’ and do the following:
- Set the value of the Embedded NIC1 enabled with PXE to true.
- Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller set’ to Off.
- Go to ‘Serial Communication’ set the value to On without Console Redirection.
- Go to ‘External Serial Connector’ set to Remote Access Device.
- Go to ‘System Security’ set to AC Power Recovery On.
- Go to ‘Report Keyboard Errors’ set to Do not Report.
Save the Changes and Exit.
Edit the XML to allow the Manager to reach the other servers in the network
In the root folder in cnet directory, configfeilvbin folder contains feilvbin.xml, a file being used by the Manager to access all the servers in the Game Server network. This XML file has been customized to fit the requirements of LU! PH. To edit the file, follow the procedures below:
Move the files – package and the XML from cnet to the root directory by following the
procedure listing below:
- Copy the new package from VMWare PC (laptop
scp root@VMWareIP:/export/cnet.feilvbin.tar.gz /root
Supply the the root password
- If the Manager does have an IP yet, run this command:
ifconfig eth0 IP
where IP is the IP Address of the Manager.
- Extract the cnet,feilvbin folder using this command
tar –zxvf cnet.feilvbin.tar.gz – uncompress the cnet.feilvbin folder.
- Go to the extracted directory and copy the archive to the root directory.
cd cnet.feilvbin
cp cpackage.feilvbin.tar.gz /root – must be in /root/cnet.feilvbin/configfeilvbin directory
- Go to the new directory of the archive and extract the archive.
cd
tar –zxvf cpackage.feilvbin.tar.gz
- Move the package to the cpackage directory
mv cpackage.feilvbin cpackage
- Edit the script that mounts the non-disk servers to the Manager by running this command
vi remount.sh
Remove the servers that do not belong to the group. Note that this script can be found at root cpackage/tool.
- Edit feilvbin.xml and input the necessary values. Refer to the documentation of the XML to determine these fields. At root directory, copy the feilvbin.xml to cpackage.
cd cnet.feilvbin
cp feilvbin.xml /root/cpackage/
vi feilvbin.xml
After reboot, execute the following commands:
cd /etc/sysconfig/network-scripts/
mv ifcfg-ppp0 ofcg-pp0.old
This is used to disable VPN access. For LU! setup, this may be skipped since
Install the package by executing this command:./package feilvbin.xml install.
Reboot the Server.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and create the partitions and format the directories of both Database and Backup:
Since both Backup and Database have only two partitions – swap and export. The mapping of the virtual directory is specified below:
sda – swap
sdb – export
Format the directories using the following commands:
For Database server:
rlogin database
mkswap /dev/sda
mkfs.xfs /dev/sdb
exit
For Backup server:
cp /sbin/mkfs.xfs /export/backup/sbin/
rlogin backup
mkswap /dev/sda
mkfs.xfs /dev/sdb
exit
Check if the partition exist, type the command “cat /proc/partitions” (view the partitions in the server)
Reboot the server.
Edit the Non-Disk Servers
For non-disk servers, only the BMC and the BIOS gets configured during setup. Refer to the configuration setup for BMC and BIOS for Database and Backup. After the configuration, reboot the servers from the Manager window by issuing the command:
ipmish –ip 192.168.1.111 –u “root” –p “password” power cycle
This command will restart the server. To check if the server is already up, do the following:
ping game 1 2 3, link, delivery,database, backup
login into the server by typing rlogin game1
type ifconfig
type ethtool eth0
Publish the Game
After setting up the server, the game should now be installed. To install the game, issue this command on the Game Manager Server:
cd cpackage
./package feilvbin.xml gameinstall.
Reboot the computer after the installation is complete.
Provide Access to the Game Server Setup
To provide client PCs access to the Game Manager, the certificates should be installed in these computers. Certificates have been provided by PW (refer to the cnet.doc in Appendix B). This access is through the web using this URL:
To be able to activate this page on the Game server, add the settings to the environment variables in the profile (.bash profile). Do the following:
vi .bash_profile
PATH=$PATH:$HOME/bin:/usr/local/jakarta-tomcat-5.5.9/bin
Export JAVA_HOME=/usr/java/jdk1.5.0_05
Log out to reload the system and use these variables.
Remove the old java program and install the new java provided by PW.
cd /usr/
mv java java.bak
cd /root/cnet.feilvbin/
cp java.tar.gz /usr
cd /usr
tar zxvf java.tar.gz
Run the Tomcat Java program:
cd /usr/local/jakarta-tomcat-5.5.9/bin
./startup.sh
Note: The latest .bash_profile given by PW already contains this modification so this step can be skipped.
II. AU Server Group
AU Server group contains the account and billing data as well as the web services that will be used for the game. To set up this server group, do the following:
- Configure the AU Manager
- Configure AUTH
- Update configuration and install the package
- Configure java in AUTH and AU Manager
- Configure SSL
- Configure Hosts
- Setup Account and Billing DB
- Configure tomcat
- Configure Account and Billing DB
- Reconfigure AU Manager and AUTH Servers
- Check the connection of AUTH and the Account and Billing Database
AU Manager is the server responsible in managing the Account and Billing DB as well as the web server. To configure the manager do the following:
RAID Configuration
DRAC Configuration
BIOS Configuration
Server System Installation
Configuration of the Boot Loader
Reconfigure BIOS to boot from hard disk
RAID Configuration
Press CTRL+R to load the RAID BIOS configuration interface.
Select Controller 0. Press F2 then select ‘Reset Config’ to delete the existing configuration.
The interface will display ‘No configuration present!’. Press F2 and select ‘Create New VD’ to create a new configuration.
The Virtual Disk Management screen will be loaded and do the following:
On the RAID Level field, choose ‘RAID-1’ and Press ‘Enter’.
Cross out the squares corresponding to the two drives in this partition by pressing space bar. The first drive is the root, second drive is the swap and the third is export.
First, set the Basic Settings for the Root.
Root:
VD Size: 10000 MB
VD Name: root
Cross out the ‘Advanced Setting’ and set the following values for the partition.
Element Size: 64 MB
Read Policy: Adaptive Read
Write Policy: Write Back
Cross out Initialize by pressing spacebar.
Create the next two partitions by going to ‘Space Allocation’ under free space and press F2. Repeat steps iii to v. Use the following values for the corresponding partitions.
Swap:
VD Size: 4096 MB
VD Name: swap
Export:
VD Size: whatever the remaining space is
VD Name: export
Press Esc and click OK to save.
Press CTRL+ALT+DEL to reboot the server.
DRAC Configuration
Go to the Remote Access Configuration by pressing CTRL+E after reboot.
Change the values of the following:
IPMI Over Lan – On
NIC Selection – Dedicated
LAN Parameters
Ethernet IP Address (specify the internal IP the IDC)
Default Gateway (specify the internal IP of the gateway)
Host Name String (input ‘Manager’)
LAN User Configuration
Enter Username: root
Enter Password: <input password here>
Confirm Password: <input password here>
Press Escape and Save Changes and Exit. The server will reboot.
Note: While doing the configuration, take note of the IP addresses that are being assigned to the servers. These will be needed later in the modification of the XML.
BIOS Configuration
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
Go to ‘Integrated Devices’ and do the following:
Set the value of the ‘Embedded NIC1 enabled with PXE’ to true.
Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller’ set to Off.
Go to ‘Serial Communication’ set the value to On without Console Redirection.
Go to ‘External Serial Connector’ set to Remote Access Device.
Go to ‘System Security’ set to AC Power Recovery On.
Go to ‘Report Keyboard Errors’ set to Do not Report.
Save the Changes and Exit.
Server System Installation
After the configuration of the server, the system will now be installed. After reboot, login to the server using root and follow the procedures below to install:
Create partitions and Format Directories.
Before creating the partitions, check if there are any existing partitions. To do this type this command cat /proc/partitions. This will display the virtual directories and the partitions for each directory. The mapping of the virtual directories are as follows:
sda – root
sdb – swap
sdc – export
If there are no partitions, proceed to the next step.
Run fdisk /dev/sda to create partition on the first virtual directory.
Press p to check the original partition of the directory.
Press n to create a new partition.
Press p to create a primary partition. Press 1 to use the default partition value.
Press p again to view created partition. This should display the device boot /dev/sda1.
Press w to save the changes in the partition table.
Format swap by executing this command – mkswap /dev/sdb.
Format sda and sdc by executing these commands:
mkfs.xfs /dev/sda1 –f
mkfs.xfs /dev/sdc -f
Mount the Partitions
Go to cd /tmp to create a new directory for sda and sdc. Issue the following commands:
cd /tmp
mkdir sda1 sdc
Mount the directories created by executing the following commands:
mount /dev/sda1 /tmp/sda1
mount /dev/sdc /tmp/sdc
Extract Image to the Manager’s Disk
Go to the directory where the image is located. Run this command – cd /dump.
Dump folder contains the archived files for billing and the games. The billing-related archives are the files with this filename – OpenBeta_AU_ *. The game-related archives are the ones with filename – hw_game_*. Notice that for each archive groups, there are files with export and root on their filenames. The archive with root on the filename consists of files used for system management. The one with export on the filename consists of files that are used for the system that manages all the other non-disk servers.
Copying of files may create inconsistencies between the files to check this issue the following commands:
cat Release_AU_root_JP.dump.gz.md5
md5sum Release_AU_root_JP.dump.gz
This step is optional and is used only to check the files.
Extract the archives to the corresponding directories by issuing the following commands:
zcat Release_AU_root_JP.dump.gz | xfsrestore - /tmp/sda1
zcat OpenBeta_AU_export.JP.dump.gz | xfsrestore - /tmp/sdc
This operation should return this message: RESTORE STATUS: SUCCESS. If this is not the return message, redo the process from the creation of the partitions.
Configuration of the Boot Loader (GRUB)
After the extraction of the image file, configure the boot loader. To do this, follow the commands below:
- Type grub to go to the interface of grub.
- Type setup (hd0) (hd0,0).
- Type quit.
- Type reboot.
To check if the kernel installed is the one that’s needed, do the following commands before rebooting the system:
cd /tmp/sda1/boot/grub
cat menu.lst
To avoid kernel panic after the aumanger is rebooted, edit the grub file default value to 0. From /root directory type the command:
vi /tmp/sda1/boot/grub/menu.lst
default=0
On this file, the version of the kernel should be equal to 2.6.17.11.management.1950.
Reconfigure BIOS
After the reboot do the following:
- Go to the BIOS Configuration and choose Boot Sequence. Uncheck Embedded NIC 1 MBA Version… option and check Hard drive C:.
- Then go to Integrated Devices, choose the Embeded GB NIC1. Change the value from Enabled with PXE to Enabled without PXE.
- Save Changes and Exit. The server will reboot. The default password of root in this server is 1234.
To configure AUTH, do the following procedures:
- Configure BIOS
- Configure BMC
- Install the package via network.
Configure BIOS
After reboot, press F2 to go to the BIOS Configuration Screen and do the following:
- Go to ‘Memory Information’ and choose ‘System Memory Testing’. Set the value of this to Disabled.
- Go to ‘Boot Sequence’. Uncheck everything except Embedded with NIC…
- Go to Integrated Devices and do the following:
- Set the value of the ‘Embedded NIC1 enabled with PXE’ to true.
- Take note of the MAC Addresses of Gb NIC1 and Gb NIC2. These MAC Addresses will be used to update the XML file. Edit the ‘IDE CD-ROM Controller’ set to Off.
- Go to ‘Serial Communication’ set the value to On without Console Redirection.
- Go to ‘External Serial Connector’ set to Remote Access Device.
- Go to ‘System Security’ set to AC Power Recovery On.
- Go to ‘Report Keyboard Errors’ set to Do not Report.
- Save the Changes and Exit.
The configuration of BMC is basically the same as DRAC since it serves the same purpose only that BMC can not be accessed using an external IP. Change the following selections to the corresponding values:
NIC Selection – Shared
LAN Parameters:
Ethernet IP Address (check the IP Address assignments in the Appendix section of this IP)
Host Name String (input ‘AUTH’)
LAN User Configuration:
Username – default to root
Password: <input password>
Confirm Password: <input password>
Install Package via Network (Aumanager)
To install the package via network, run the following commands:
scp root@laptopid:/export/cnet.feilvbin.tar.gz /root
tar –zxvf cnet.feilvbin.tar.gz
Update configuration and install the package.
Edit aumanager.xml. Change auth kernel to vmlinuz-2.6.17.11.nodisk1950,
After editing, copy the kernel to pxeboot by executing the following commands:
cd /boot
cp vmlinuz-2.6.17.11.nodisk.1950 /tftpboot/linux/pxeboot
cd /root/cnet.feilvbin/
cp aumanager.xml /root/package_only_for_au/config/
cd /root/package_only_for_au
Install the package by running this command:
./package config/aumanager.xml install
reboot
Configure java in the AUTH and AU Manager
Configure java in AU Manager:
Log in to AU Manager Server as root.
Go to the cnet directory – cd /root/cnet.feilvbin
Remove the current installation of java.
rm –rf /usr/java
.
Copy the archived java setup files to the usr directory – cp java.tar.gz /usr/.
Go to the usr folder – cd /usr/.
Extract the archive to this directory by executing this command - tar zxvf java.tar.gz. This will install the jdk to the server. The jdk version that is being used for Perfect World is 1.5.0_05.
This will extract the jdk to the server.
Go to the library of java directory – cd java/jdk 1.5.0_05/lib/.
Copy the bash profile from the cnet to the server, Run these commands:
cd /root/cnet.feilvbin
cp bash_profile /root/.bash_profile
ipmish –ip authBMC IP –u “root” –p “password” power cycle
Configure java in AUTH:
Go to the cnet directory – cd /root/cnet.feilvbin
Remove the current java installation.
rm –rf /export/auth/usr/java
Copy the archived java setup files to the usr directory of auth – cp java.tar.gz /export/auth/usr/.
Go to the usr folder – cd /export/auth/usr/.
Extract the archive to this directory by executing this command - tar zxvf java.tar.gz. This will install the jdk to the server. The jdk version that is being used for Perfect World is 1.5.0_05. This will extract the jdk to the server.
Copy the files from cnet to the server. Run the following commands:.
cd /root/cnet.feilvbin
cp bash_profile /export/auth/root/.bash_profile
Configure the SSL
Remove any existing SSL configuration: rm –rf /etc/ssl.
Copy the SSL archive from cnet.feilvbin to etc – cp ssl.tar.gz /etc/
Go to etc folder and extract the archive.
cd /etc/
tar zxvf ssl.tar.gz
Configure Hosts
Edit the host file aumanager – vi /etc/hosts. Entries in the file should look like this:
x.x.x.254 aumanager
x.x.x.1 auth
x.x.x.10 audb
The IP should be the internal address of the servers. Save and Exit.
Edit the host file in auth – vi /export/auth/etc/hosts/. Entries in the file should look like this:
x.x.x.254 manager
x.x.x.1 auth
x.x.x.10 audb
Login to auth and check the configuration. Run the following commands:
rlogin auth
ifconfig
Set Up Account and Billing DB (audb)
- Move the following files: mssql_table.sql and mssql_storedprocedure.sql.
- Configure the XML. Specify the IP and the port of the account database. Change also the database name, username and password on the connection.
vi table.xml
cd /root/cnet.feilvbin/configfeilvbin
cp table.xml /etc/
cp table.xml /export/auth/etc/
- Copy gmopen.xml to aumanager and auth. Gmopen.xml contains the server list.
vi gmopen
cp gmopen.xml /etc/
cp gmopen.xml /export/auth/etc/
- Copy authd.conf from cnet to /export/auth/.
cd /root/cent.feilvbin/configfielvbin
cp authd.conf /export/auth/etc/
No need to edit this file, Perfect World will provide a customized authd.conf file to cater to the needs of LU! Philippines.
- Edit the GMServer.conf. Change the IP address specified in the file to the address of the GM Tool. Copy the file to the aumanager and auth from cnet.
vi GMServer.conf
Save and Exit.
cp GMServer.conf /etc/
- Copy the caurhtd.tar.gz to auth and extract the contents in the directory.
cp cauthd.tar.gz /export/auth/root/
cd /export/auth/root/
tar zxvf cauthd.tar.gz
This creates a folder cauthd in the auth server.
- Login to auth and run ./authd & then Exit.
Bash:
rlogin auth
cd cauthd/build
./authd &
exit./authd & is used to boot AUTH. The result of running this command should be authd: add PollIO task. When this is the output, it means that the boot is successful.
Configure tomcat
Go to the webapps directory in aumanager and clear the contents of the folder. Execute the following commands:
Bash:
rm –rf /usr/local/jakarta-tomcat/5.5.9
cd /root/cnet.feilvbin
cp jakarta-tomcat-5.5.9.aumanager.tar.gz /usr/local
cd /usr/local
tar zxvf jakarta-tomcat-5.5.9.tar.g
killall -9 javaIssue the following commands to start tomcat.
cd /usr/local/Jakarta-tomcat-5.5.9/bin
startup.sh
To check if the tomcat is up, access the webmanager interface – https://x.x.x.254/monitor/ where x.x.x.254 if the internal IP of the aumanager.
Configure the Account and Billing Database
The Account and Billing DB uses Windows 2003 for OS and MS SQL Server 2000 Enterprise Edition.
Open the .sql files (mssql_table.sql and mssql_storedprocedure.sql) and run in the Query Analyser to build the database. Mssql_table.sql creates the database and the tables while mssql_storedprocedure.sql creates the stored procedure being used by the system. Remember delete all the comments before running the commands in the Query Analyzer.
Once everything has been created, create user to access the database.
account: fielvbin
db: audb
permission: dbowner
Reconfigure AU Manager and AUTH Servers
Go to AU Manager and edit table.xml.
cd /etc/
vi table.xml
Change the IP of the database, database name, username and password. Save and exit.
Go to AUTH and edit table.xml
vi /export/auth/etc/table.xml
Change the IP of the database, database name, username and password. Save and exit.
Reboot AUTH – ipmish -ip 192.168.1.100 -u “root” -p “password” power cycle.
After reboot, run authd on AUTH Server and exit. Execute the following commands:
Bash:
rlogin auth
cd cauthd/build
./authd &
exitGo back to the AU Manager and restart tomcat. Execute the following commands:
Bash:
killall -9 java
ps –aux | grep java
/usr/local/Jakarta-tomcat-5.5.9/bin/startup.shCheck the connection of AUTH and the Account and Billing Database
To check the connection between AUTH and the Account and Billing Database, access the webmanager system.
Access http://aumanagerip/monitor via web. This is the tool that checks the connection of the aumanager and the audb.
A popup screen will be displayed, choose iwebmanage. Accept the certificate to enter the site.
Try to register a new user. Verify by checking the existence of the record on the database.
Note: If there are any errors, check the logs of the server using this command: tail -f /usr/local/jakarta-tomcat-5.5.9/logs/catalina.out.
III. Patch Server Group
Perfect World did not provide an installation package for the patch server, installation will be done remotely by PW. Below is a summary of the work that needs to be done by Level Up! and Perfect World to set up this server group:
Level Up! will:
prepare the servers, install necessary software – OS (Fedora Core2), apache, FTP
install firewall in this setup
open these servers to Perfect World so they will have access.
If one of the servers goes down, LU! just has to reinstall all the necessary software and seek PW for support.
Perfect World will:
configure servers and install the clients that will be used for the upload and download of the patch.
In case one of the patch servers goes down, PW will provide support and reconfigure the patch server.
IV. GM Server
GM Server contains the GM Tool, the web application used by Customer Support. The web application and the database can be hosted in separate servers but in this document it is assumed that they will be hosted on the same server. The GM Server communicates with AU and iweb as illustrated below:
To access the tool the user must have a certificate stored in USB drive. This certificate is created and issued to users who need to access the site.
To access the tool:
- Plug in the USB containing the certificate in the computer.
- Go to the URL of the GM Tool. The certificate authentication will be displayed.
- When the page is loaded, input username and password.
- When login is valid, user is taken to the GM Tool. The functions available on the tool depend on the access level of the user. Access level is defined when the user is being created in the tool.
For LUPH implementation, the certificate will not be used. So the for the access of this server, straight forward username and password validation through the database will be used.
To setup the server, do the following:
- Install the necessary software to run and host the GM Tool – Fedora Core 2, tomcat, jdk, mysql. The tomcat that should be installed on this server should be the same tomcat installed in AU Manager. Get this installer from the auth package.
- Setup the db by running the script provided by Perfect World – csm.sql. Remember to check first if the database already exists before running the script.
- Copy WebModule to tomcat/webapps/ (usr/local/jakarta-tomcat-5.5.9/webapps).
- Modify server configuration file – server.xml
XML:
<context path=”/gm” docBase=”<web application path>” debug=”0” reloadable=”true”>
<resource name=”jdbc/csm”
auth=”Container”
driverClassName=”org.gjt.mm.mysql.Driver”
maxActive=””
maxIdle=”20”
maxWait=”5000”
type=”javax.sql.DataSource”
url=”jdbc://gm server ip:3306/csm?useUnicode=true&characterEncoding=GBK”
usename=”root”
password=””/>
</context>- Modify system config files.
- Modify hibernate.cgf.xml
<property name=”hibernate.connection.url”> <!CDATA[jdbc:mysql://gmserverIP:3306/csm?useUnicode=true&characterEncoding=GBK&autoReconnect=true]]>
</property>
<property name=”hibernate.connection.username”>dbusername</property>
<property name=”hibernate.connection.password”>dbpassword</property>
where:
gmserverIP – pertains to the IP of the server where the GM Tool is deployed.
dbusername and dbpassword – are the username and password used by mysql.
- Modify wsurl.properties.
webservice
url = https://aumanagerIP:443/GMServer/gmservice
userurl = https://aumanagerIP:443/GMServer/userservice
where:
aumanagerIP – pertains to the IP of the au manager server
- Modify log4j.conf to modify the location of the log output file.
- Modify ApplicationsResources_zh_CN.properties to modify the language settings. After editing this file, do the following:
Rename the file to csmInfo.properties.
- Modify menus.xml if there’s a need to modify the menu of the web application.
- Modify CoreConfig.xml if there’s a need to modify the system access levels.
- Overwrite existing web.xml with the new web.xml which was given by PW.
V. Log Server
The logs are generated in the Game Server (/var/log). These logs are as follows:
brief.tar.gz
contains the character and faction information
downloaded from the Manager(game) to the Log Server
archive contains CSV files which are dumped into the database
tables that get populated with this data are: chardata, faction, factionuser
logs can be viewed using the web application for the logs – http://logserverIP/logs/defaul.htm
retention period: 30 days
world2.formatlog.1.gz
contains the game details of the accounts.
downloaded from the Manager(game) to the Log Server.
archive contains CSV files which are dumped into the database
sample tables that gets filled up by this data are: AccountLog, RemoteLog, UpgradeLog
logs can be viewed using the web application for the logs – http://logserverIP/logs/default.htm
retention period: 30 days
world2.err.8
contains the server error logs
not downloaded to the Log Server
user needs to read through the log to check the details because this can not be viewed through web.
retention period: 15-30 days on the Game Manager.
world2.chat.1.gz
contains the chat logs
not downloaded to the Log Server
can be queried using a web interface – http://logserverIP/kchat/index.cgi
retention period: 15-30 days on the Game Manager.
world2.log
contains the character details
can be queried using a web interface – http://logserverIP/kcfgi/index.cgi
Among these logs, only brief.tar.gz and world2.formatlog.1.gz are downloaded to the Log Server. The download happens at 4:00 AM. These logs are analyzed and dumped into the database in the Log Server at 5:05 AM. Note that if there is more than one server group, the download of the backup maybe done simultaneously. But it should be ensured that the download of the logs finish before the job to dump these logs into the database starts.
Log Server has no installation package. For the setup the following items are the summary of the work that needs to be done by Level Up! and Perfect World:
Level Up! will:
prepare the server, install necessary software – Fedora 4
create the following partitions
File System capacity
/dev/sda2 40G
/dev/sda1 99M
tmpfs 1011M
/dev/sda5 236G
allow and give the IP of the server to Perfect World for remote access
Perfect World will:
deploy the Log Tool on the server
provide remote support
Operations
Restarting the System
When restarting the servers in the network, the proper sequence to be followed is:
- Restart Manager
- Restart Delivery
- Restart DB
- Restart Backup
- Restart Game Servers
- Restart Link
Monitoring is done using cricket in iweb. Cricket is a system used to monitor trends in time-series data. It helps the network managers visualize and understand the traffic on their networks. To access this tool, go to https://gamemanagerIP/iweb/monitor and click on the Monitoring Information link. The tools available for this link are as follows:
Cricket
Syslog
Route (trace route)
Views (monitoring graphs)
The graphs displayed on this page are as follows:
- Delivery Task Queue
- Online Users
- Link – eth0 octet (bits/sec)
- Link – eth 1 octet (bits/sec)
- Link1 tcp connections
- Link2 tcp connections
- Beijing ping time
- Shanghai ping time
- Link1 CPU
- Link1 Memory
- Link2 CPU
- Link2 Memory
- Game1 CPU
- Game1 Memory
- Game2 CPU
- Game2 Memory
- Game3 CPU
- Game3 Memory
- Delivery CPU
- Delivery Memory
- Database CPU
- Database Memory
- Backup CPU
- Backup Memory
- Manager CPU
- Manager Memory
- Database IO Stat
- Database Storage Space
- Backup IO Stat
- Backup Storage Space
- Manager IO Stat
- Manager Storage Space
- NFSRoot IO Stat
- NFSRoot Storage Space
Maintenance
Game Server Maintenance is done primarily by using the web management tool.
