|
|
 | I also have a blog for your reading pleasure. |
RHQ Notes
- Here's some SQL to get the partition table (the agents' failover lists):
select s.name as server, a.name as agent
from rhq_server s, rhq_agent a, rhq_failover_list fl, rhq_failover_details fd
where a.id = fl.agent_id AND fd.ordinal=0 AND fd.failover_list_id=fl.id AND fd.server_id=s.id
order by s.name, a.name
- To build a distro (of RHQ and Jopr)
- Do a full RHQ distro build within rhq/trunk via mvn -Penterprise,dist -Dmaven.test.skip install
- Now build Jopr within jopr/trunk via mvn -Pdist -Dmaven.test.skip install
- Jopr distros are found here: jopr/trunk/modules/dist
- Here's some JPQL/SQL for the admin/hibernate.jsp page:
- OOBs For Definitions Across all resources of a type
select count(oob), sched.definition.id, sched.definition.resourceType.name
from MeasurementOutOfBounds oob join oob.schedule sched
group by sched.definition.id, sched.definition.resourceType.name
order by count(oob) desc
- OOBs for individual resources/metrics
select count(oob) as count, res.id as resource_id,
sched.id as schedule_id, res.name as resource_name,
sched.definition.name as metric_name
from MeasurementOutOfBounds oob join oob.schedule sched join sched.resource res
group by res.id, sched.id, res.name, sched.definition.name
order by count(oob) desc
select count(measuremen0_.id) as col_0_0_, resource2_.ID as col_1_0_,
measuremen1_.id as col_2_0_, resource2_.NAME as col_3_0_, measuremen3_.name as col_4_0_
from RHQ_MEASUREMENT_OOB measuremen0_
inner join RHQ_MEASUREMENT_SCHED measuremen1_ on measuremen0_.SCHEDULE_ID=measuremen1_.id
inner join RHQ_RESOURCE resource2_ on measuremen1_.RESOURCE_ID=resource2_.ID,
RHQ_MEASUREMENT_DEF measuremen3_
where measuremen1_.DEFINITION=measuremen3_.ID
group by resource2_.ID , measuremen1_.id , resource2_.NAME , measuremen3_.name
order by count(measuremen0_.id) desc
- If you want to test connectivity to a server or agent:
- cd jbossas/server/default/deploy/rhq.ear/lib
- To ping a server, the -u value should be servlet://server-host:7080/jboss-remoting-servlet-invoker/ServerInvokerServlet
- To ping an agent, the -u value should be socket://agent-host:16163
Git Notes
- Config
- Setup global user info in ~/.gitconfig for commit details
- core.autocrlf=input makes sure you checkin text files with LF EOL chars (strips CR)
- diff.external=/home/mazz/opt/git/diff.py where diff.py launches Meld, provides a nice diff GUI as opposed to reading diff files.
- git config help: http://www.kernel.org/pub/software/scm/git/docs/git-config.html
- Tracking files
- git tracks files you need to commit - it stores this information in the "index"
- Add all files, recursing down subdirectories, to the index. This effectively stages the files that can now be committed.
- Get information on what files are ready to be commited and what files are "unstaged" (aka not yet 'git added')
- Cloning / Remote Repositories
- Clone a local working copy to a bare .git directory repo
- git clone --bare /path/to/local/repo repo.git
- Clone a remote git repo so you can begin working on it
- git clone <remote git URL>
- To revert a change back to what was checked in (i.e. back out modifications to a working copy)
- git checkout -- <files to revert>
- To store the repo on a remote mirror
- First you must create a remote "mirror" definition on your local repo (you must cd to the repo root):
- git remote add your-remote-id username@hostname:/your/location/to/repo.git
- Now after you commit things to your local repo and you want to push up to the remote, you do this:
- git push --mirror your-remote-id
- GUI Tools
- To peruse the repo, branches and working copy, use the GUI tool
- The GUI tool has menu options to get to the gitk tool, but you can get it by gitk also
- .gitignore
- To ignore files, you can use .gitignore.
- Patterns are read from a .gitignore file in the same directory as the path, or in any parent directory, with patterns in the higher level files (up to the toplevel of the work tree) being overridden by those in lower level files down to the directory containing the file.
- These patterns match relative to the location of the .gitignore file.
- Logs
- To get a log of history
- git log -—pretty=<oneline|short|medium|full|fuller|email|raw|format:string>
- There are other options to the git log command that let you filter the results
- --since="1 month ago" --until=yesterday --author="mazz"
- Tons more options, including diff history. See http://www.kernel.org/pub/software/scm/git/docs/git-log.html
- Diff
- git diff is how you generate patch files and look at diffs. See http://www.kernel.org/pub/software/scm/git/docs/git-diff.html
- If you want to see what you would be committing (i.e. the diff between HEAD and the index):
- What is different between the index and the local working copy files
- git diff-files --name-status tells you just what files are changed
- git whatchanged -p shows a patch diff of past history
- git reflog shows you when the tip of the branch changed
- git diff origin/master..master tells what I changed from the remote repo
- If you setup an external diff viewer, this provides a GUI editor of the diff
- gitk origin/master..master will show the diff in gitk
- git diff --no-ext-diff disables the external diff viewer
- If you have two branches A and B, both started at a same common commit, but since then both have diverged differently, you can use the "triple dot" notation to see how they both diverged:
- If I don't want to pull from the remote, but I want to see what changed up there:
- git fetch origin <branch>
- git diff --name-only origin/<branch>
- git diff origin/<branch>
- Checkout
- To checkout an old version (this example gets the one from the previous commit):
- git checkout HEAD^ path/to/file
- git show HEAD^ path/to/file just shows the file, but doesn't check it out
- Patches
- Create a patch for all commits that are different from origin that can be emailed
- git format-patch --stdout origin
- Create a patch for the last commit just performed on the current branch
- git format-patch --stdout HEAD^1..HEAD
- Apply a series of patches from emails
- cat ...output from format-patch... | git am
- Apply a diff to working copy and index (not committed)
- git apply ...diff.patch...
- Branching
- To see what local branches are in the repo, and which one your working copy is currently on:
- To create a new branch
- git branch <name of new branch>
- To switch to a branch
- git checkout <name of branch>
- How to checkout a "tracking" branch that tracks a remote branch which you can push to/pull from
- git checkout --track -b branch_name origin/branch_name
- How you can create your own remote branch and begin working in it
- To diff between two branches
- git diff --stat <branch1> <branch2>
- To delete a branch (usually done after you merge into master and no longer need the branch)
- git branch -d <name of doomed branch>
- To delete a remote branch (dangerous!)
- git push origin :doomed-branch-name
- To merge a branch, you checkout the one branch, and ask to merge the second
- git merge <name of branch to merge into working copy>
- git reset allows you to revert back if a merge has too many conflicts
- Rebasing means you just move the base of your working copy to a given branch
- You must have everything added/committed to your index
- git rebase <branch to base the working copy on>
- If you are working on something and don't want to commit to your branch, but need to switch to another branch, you can stash your changes. After you switch to the other branch, finish doing what you need and switch back, you can apply your stash getting back to the way you were before
- git stash
- git stash apply
- You can see what you stashed using the options "list" and "show" to git stash
- You can peek at what a remote repo changed without merging first, using the "fetch" command; this allows you to inspect what someone else did, using a special symbol "FETCH_HEAD", in order to determine if there is anything worth pulling, like this:
- This operation is safe even if you have uncommitted local changes. The range notation "HEAD..FETCH_HEAD" means "show everything that is reachable from the FETCH_HEAD but exclude anything that is reachable from HEAD".
- If you want to visualize this: gitk HEAD..FETCH_HEAD
- Tagging
- You can tag a branch, usually to denote a release
- git tag -a <tag name>
- TODO: how do you switch to or checkout a tag???
- Maintenance
- You need to garbage collect the metadata periodically via git gc
- Use git fsck to see if you have unreachable or corrupted objects
- Use git prune to clean up problems that fsck finds
- git update-index --refresh after you copy a repo to make sure the index is OK
Java Notes
- Running JPDA debugging from command line:
- To run a VM with JMX remoting enabled without authentication, you must pass in the following environment variables:
- -Dcom.sun.management.jmxremote.port=19988
- -Dcom.sun.management.jmxremote.ssl=false
- -Dcom.sun.management.jmxremote.authenticate=false
- To run a VM with JMX remoting enabled with password authentication:
- -Dcom.sun.management.jmxremote.port=19988
- -Dcom.sun.management.jmxremote.ssl=false
- -Dcom.sun.management.jmxremote.authenticate=true
- -Dcom.sun.management.jmxremote.password.file=/some/directory/jmxremote.password
Note that "jmxremote.password" must be read-only. On Windows, you must use "cacls" command to do this: cacls /some/directory/jmxremote.password /P username:R
A password file template is located at $JRE_HOME/lib/management/jmxremote.password.template.
There is also an auth file that you can use to define other roles.
For more information on setting this up and setting up SSL, see http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html.
- To run JBossAS 4 with JMX remoting enabled and with platform MBeans exposed via JNP too, pass in these extra variables:
- -Djboss.platform.mbeanserver
- -Djavax.management.builder.initial=org.jboss.system.server.jmx.MBeanServerBuilderImpl
Oracle Notes
- To get stats, like processes and sessions: select * from V$RESOURCE_LIMIT
- To enable recovery on Oracle, the user must have permissions such as these
grant select on sys.dba_pending_transactions to username;
grant select on sys.pending_trans$ to username;
grant select on sys.dba_2pc_pending to username;
grant execute on sys.dbms_system to username;
grant select on v$xatrans$ to username;
grant execute on dbms_system to username;
Postgres Notes
RHQ Test Scripts
- I wrote some simple Windows batch scripts that I use when testing on my box.
- Here's the page on what scripts to run for the agentspawn stuff: Design-AgentSpawn
- Here is a test script used to run the agentspawn stuff on multiple boxes that have an NFS directory on mulitple servers. Create an "agents" directory and put the following script in it, named runspawn.sh:
#!/bin/sh
if [ "$2" == "" ]; then
echo "Syntax: $0 <hostname> <copy|start|stop|clean|help>"
exit 1;
fi
echo Copying $1 agentspawn.properties file
cp properties-files/agentspawn.properties.$1 agentspawn/src/scripts/agentspawn.properties
echo Running ant $2
cd agentspawn/src/scripts
ant $2
Checkout or copy the agentspawn module in here. Then create a directory called properties-files and in there put copies of agentspawn.properties in there (name them agentspawn.properties.<hostname>. Each of those properties files can be customized for the agents that will be spawned on that host (you probably will want to change the vm.dir and the starting ports). Now you can run that script, giving it the hostname as the first argument and the ant target as the second. Here's what the file/directory layout should be:
(note: The agentspawn is the full contents of the /etc/agentspawn module, including the /target binaries after you "mvn install" it).
- Here's test scripts I use to prepare multiple server distros when each box has the same NFS mount. You have to first build the software, e.g.:
This produces "rhq/trunk/dev-container". Now you can run copy-servers.sh to copy the distros for the different hosts. delete-servers.sh will purge all the distros. run-server.sh will run the proper distro, assuming your server distros are put into directories with the same name as their host (i.e. "hostname -s").
echo Making server directories to host the software for all servers
mkdir jon01
mkdir jon02
mkdir jon03
mkdir jon05
echo Copying the jon01 container
cp -R ~/source-code/rhq/trunk/dev-container/* ./jon01
echo Copying the jon02 container
cp -R ~/source-code/rhq/trunk/dev-container/* ./jon02
echo Copying the jon03 container
cp -R ~/source-code/rhq/trunk/dev-container/* ./jon03
echo Copying the jon05 container
cp -R ~/source-code/rhq/trunk/dev-container/* ./jon05
echo Copying the server configuration file to all servers
cp rhq-server.properties.jon01 ./jon01/bin/rhq-server.properties
cp rhq-server.properties.jon02 ./jon02/bin/rhq-server.properties
cp rhq-server.properties.jon03 ./jon03/bin/rhq-server.properties
cp rhq-server.properties.jon05 ./jon05/bin/rhq-server.properties
echo Copying the perftest plugin
if [ -f ./rhq-perftest-plugin-2.1.0-SNAPSHOT.jar ]; then
cp ./ rhq-perftest-plugin-2.1.0-SNAPSHOT.jar ./jon01/jbossas/*/rhq-downloads/rhq-plugins
cp ./ rhq-perftest-plugin-2.1.0-SNAPSHOT.jar ./jon02/jbossas/*/rhq-downloads/rhq-plugins
cp ./ rhq-perftest-plugin-2.1.0-SNAPSHOT.jar ./jon03/jbossas/*/rhq-downloads/rhq-plugins
cp ./ rhq-perftest-plugin-2.1.0-SNAPSHOT.jar ./jon05/jbossas/*/rhq-downloads/rhq-plugins
else
echo PERFTEST PLUGIN NOT FOUND HERE - IT WILL NOT BE DEPLOYED
fi
echo Done.
echo Removing all server directories
rm -rf jon01
rm -rf jon02
rm -rf jon03
rm -rf jon05
echo Done.
#!/bin/sh
_host=`hostname -s`
cd ${_host}/bin
./rhq-server.sh $*
Maven Notes
SVN Notes
Here's some useful tidbits regarding SVN usage.
- Suppose we build a GA distribution from trunk (svn rev 100) and create a tag off of svn rev 100 called 1.0.GA. We've since moved forward on trunk with lots of new commits - we are up to svn rev 300 now. Now suppose we need to patch the 1.0.GA product. The fix for the patch was already committed to trunk, it was committed in svn rev 150. How can we apply the fix to the 1.0.GA.
- Do get a file that was deleted from SVN, you have to use svn log --verbose to find out which revision deleted it, then you svn up -r <rev minus one> file.txt to retrieve the file. Note that this means you need to have some working copy on your file system to do the svn log and svn up commands using these commands. You might be able to pass URLs to the commands if you don't have a working copy.
Miscellaneous Notes
SQL INNER JOIN, LEFT JOIN, RIGHT JOIN
Joins can be thought of in the context of a Venn diagram. You have two circles - A and B. They overlap in some parts.
The overlap is your INNER join - only the elements in A that also have associated B elements.
The A circle is the LEFT join - all A elements regardless of whether they have associated B elements.
The B circle is the RIGHT join - all B elements regardless of whether they have associated A elements.
Setting Time on UNIX Machines
If you have a set of UNIX machines and their clocks are not NTP-synced, use the "date" command to set their clocks to within a second or two to each other. To get the time of one machine in the format that "date" needs to set the clock, run this:
Take the result of that and run it on the rest of the machines.
Calculating Running Average
long newValue = ...;
long count = currentCountOfItems++; currentAvg = (((count - 1) * currentAvg) + newValue) / count;
Testing Multicast
If you aren't sure a box has multicast enabled, run the following JGroups test app. This will run the receiver - it will print to stdout when it hears a multicast:
Now run the sender - when this starts, enter some text and hit enter to broadcast the message to all receivers:
Play with -bind_addr to see what NICs support multicasting.
Sample Standalone Hibernate/JPA
Attached is a standalone Hibernate/JPA application based off of the Hibernate/JPA Hello World sample app. Build it with "ant package", then run the executable jar via "java -jar build/helloworld.jar". Type "help" at the prompt for the commands you can use to insert, delete and query the database. The database is Hypersonic using its in-memory feature - no database files or data will be persisted. There are two Hibernate/JPA entities - hello.Message and hello.Person. You can insert, delete and query them via the "add", "delete" and "get" commands. Note that if you want to use the RHQ hibernate plugin to manage it via jmx remoting, you have to set some system properties to configure the VM:
Sample Standalone JMX App
Attached is a very small sample JMX App. It contains a prebuilt jmxapp.jar plus the source and an ant script to build the jar. It has a "runjmxapp" script that you can use as the command to start the JMX app. All it does is register a few MBeans to the platform MBeanServer and wait until the VM is killed. One MBean's name is a hardcoded static name, and the others have names that are partially determined at runtime. Use this when you need to test connectivity to a remote JMX MBeanServer.
Convert Epoch Milliseconds To Date
Attached is a small executable jar that converts epoch milliseconds to date strings. To can convert in either direction (epoch millis to date or vice versa). Enter java -jar epochmillis.jar for usage help. You can put more than one argument on the command line to convert more than one value:
JBossTM/Arjuna Notes
- Periodically, the recovery manager will ask the resource (e.g. the database), "what transactions do YOU know of that are pending and need to be recovered?". This is because the database also has the ability to mark a transaction as "prepared and waiting for a conclusion". It asks this independently of what the resource manager itself thinks is pending - i.e. the RM tries to recovery both transactions it knows of in its object store (top-down recovery) as well as transactions the database knows of (bottom-up recovery). It needs an XAResource to ask this. This is why in Oracle you need to grant some select permissions to the datasource's user - because the database's pending transactions are stored in special dba tables and its those tables the RM looks at to determine what is pending in the database.
- There is a way to plug in a custom object store implementation. The object store stores transaction logs that include information about transactions that are in the 2PC stage. Out of box impls include database object store and filesystem object store (which is used out-of-box of JBossAS, and its stored in data/tx-object-store). You can write an in-memory store implementation - but you have to make sure we only store a finite size of data to avoid OOM or just throw the logs away (but if we do that, why bother? just don't use XA). There is no config prop to say, "don't log" - you have to write your own object store impl to do that.
- 1PC is on by default if there is a single resource enlisted in a transaction (no prepare phase), even if its XA. NO LOGS ARE WRITTEN when a single resource is enlisted.
- If two <local-tx-datasource> resources are enlisted in a transaction, does JBossTM still try to do "pseudo-XA" things with it? See https://jira.jboss.org/jira/browse/JBTM-443
- JBossMQ has its own XA implementation (which is why it has JMS_TRANSACTIONS table). We can't disable that AFAIK. Therefore, it provides its own XAResource which itself requires a JBossTM recovery module. See relevant JIRAS:
https://jira.jboss.org/jira/browse/JBTM-279
https://jira.jboss.org/jira/browse/JBAS-5502
- Performance: XA vs. Local-Tx - there are additional round trips in XA - the 2PC protocol has the prepare, et. al. phases that introduce about 3 to 4 additional round trips every transaction.
- The # of resources enlisted in a transaction matter, not what type the resources are (XA va. non-XA). If 2+ resources are enlisted, a log is written out and 2PC is attempted. If one is a local-tx, then a wrapper around its resource is provided that "simulate" an XA resource - but during the 2PC prepare phase, the prepare is a no-op in that wrapper. Note that when 2+ resources are enlisted, logs are written to the object store, even if they aren't XA.
- "SELECT formatid, globalid, branchid FROM SYS.DBA_PENDING_TRANSACTIONS" is executed every X seconds (defined by com.arjuna.ats.arjuna.recovery.periodicRecoveryPeriod which defaults to 120).
- com.arjuna.ats.arjuna.coordinator.txReaperTimeout (set to 120000) is for tx timeouts. As per "jhalliday", "actually its on-demand now, although periodic is still a config option. it wakes up, looks for live tx that have reached their allotted max lifetime and rolls them back. it actually won't touch ones that have been prepared, only the ones that have not reached that stage yet."
- If the recovery fails, the resource manager continues to attempt to recover (unless a very specific set of errors are seen, in which case the RM knows the transaction is lost forever and no longer attempts it). You should be able to configure an expiration time - when this times out, the expiration code will remove the tx logs from the object store and stop trying to recover that tx. There is a bug currently in JBossTM in which that expiration never happens: https://jira.jboss.org/jira/browse/JBTM-418
- Find out how to enable debug to trace 2PC and object store access: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=147697
Linux/Fedora Notes
- /etc/sysconfig/networking/profiles/default contains network settings
- hosts - default hosts, when you switch connections, this overwrites /etc/hosts
- ifcfg-eth0 - default settings when you switch to this device. I set PEERDNS=no which seems to tell network manager that I don't want my router to be considered a DNS server (PEERDNS=yes added 192.168.0.1 to my /etc/resolve.conf, which I did not want)
- If some network connections but not all, and everything else is eliminated, try to lower the MTU setting on the network interface (I had to do this on a virtual machine guest) :
- The "thread-max" kernel setting
- To find out what the current max threads limit is: sysctl -a | grep threads
- change the value on-the-fly with: echo [new value] > /proc/sys/kernel/threads-max
- make the setting permanent by adding "kernel.threads-max = [new value]" in /etc/sysctl.conf file and then issue the "sysctl -p" command to load it.
Useful Firefox Plugins
Here are some Firefox plugins that I've found useful.
Useful Eclipse Plugins
Useful Thunderbird Plugins
Here are some Thunderbird plugins that I've found useful.
Useful Third-Party Apps
|