Monday, December 31, 2012

Java: Store/Load properties file in both XML and TEXT format.

Most of the time we created the properties file and modifying it in text editors like Notepad++ and notpad etc. We also can create and modifies the properties files from Java program as well. You will see in the following example XmlTextPropsReaderWriter.java how to do it.

The most common properties files are Log4j.properties that is used to configure Log4J system logging, and jdbc.properties which also used to control your program JDBC API connectivity parameters, and finally no web application working with internationalization (I18N) doesn’t have a resource bundle in form of properties file.

Let's see how to read and create properties file from java program in both formats (XML and TEXT).

Java API’s java.util.Properties class provides utility store() methods to store properties in either text or xml format. store() can be used to store property in text properties file and storeToXML() method can be used for creating a property file in XML format. On the other hand the load() reads a text format properties file while loadFromXML() reads XML format properties file.

The following example is streamlined using new JDK7 feature and NIO.2 capabilities.

Program source:

Running the above program produces the following output:

value1
value3

The created properties file text (users.properties) format is:

And the created properties file XML (user.xml) format is:

Note that: xml file created is governed by the following properties.dtd specification file:

Although most of us even me by default using the normal text format of the properties file and statically defined. Then the questions that may raise here are:

1- When to use the XML format?
2- When we can benefit from creating it dynamically?

for the first question, there are many areas that you can benefit from the xml format especially when you working with the same configuration file from many systems, and working with different languages to specific devices and need one configuration understood by all languages, readable, easy to secure, and retrieving data using xPath API is doable, all of these benefits leads to xml properties format.

The second part is, whenever you want to create a dynamic configuration, then you have the solution now.


Saturday, December 22, 2012

Database: Protecting your database using Database Firewall App.!

As Systems Architect I am responsible and involved in all security aspects of the system I am Architecting, and any other integrated systems or components.

When Architecting a financial products and also being related to government, then the security becomes a crucial part. One of the most important parts of protection is to protect data that resides in database.

In old days I did it manually by coding, hashing, encrypting, preventing any SQL-injection, vulnerabilities, logging, auditing ….etc. lot of time consuming and exposed to be hacked.

Nowadays, one of the amazing product I am using, to do all of the above for me beside another tremendous features is "Oracle Audit Vault and Database Firewall".

Well, what is "Oracle Audit Vault and Database Firewall" then?

Oracle Audit Vault and Database Firewall provide a first line of defense for databases and consolidate audit data from databases, operating systems, and directories.

A highly accurate SQL grammar-based technology monitors and blocks unauthorized SQL traffic before it reaches the database. Information from the network is combined with detailed audit information for easy compliance reporting and alerting.

With Oracle Audit Vault and Database Firewall, monitoring controls can be easily tailored to meet enterprise security requirements.

It secures databases by monitoring and blocking SQL statements according to the policies you define as well as collecting and reporting various configurable database audit data.

It contains many features and tools that provide ease of administration, monitoring and scaling.


The Database Firewall for Activity Monitoring and Blocking
-------------------------------------------------------------------------
Oracle Database Firewall provides a sophisticated next-generation SQL grammar analysis engine that inspects SQL statements going to the database and determines with high accuracy whether to allow, log, alert, substitute, or block the SQL.

Oracle Database Firewall supports white list, black list, and exception list based polices.

A white list is simply the set of approved SQL statements that the database firewall expects to see. These can be learned over time or developed in a test environment.

A black list includes SQL statements from specific users, IP addresses, or specific types that are not permitted for the database.

Exception list-based policies provide additional deployment flexibility to override the white list or black list policies.

Policies can be enforced based upon attributes, including SQL category, time of day, application, user, and IP address. This flexibility, combined with highly accurate SQL grammar analysis, enables organizations to minimize false alerts, and only collect data that is important.

Firewall events are logged to the Audit Vault Server enabling reports to span information observed on the network alongside audit data.


Fine Grained, Customizable Reporting and Alerting
-----------------------------------------------------------------
Dozens of out-of-the-box reports provide easy, customized reporting for regulations such as SOX (Sarbanes-Oxley Act), PCI DSS (Payment Card Industry), GLBA (German-Leach-Bliley Act ), DPA (Data Protection Act), and HIPAA (Health Insurance Portability and Accountability Act).

The reports aggregate both the network events and audit data from the monitored systems. Report data can be easily filtered, enabling quick analysis of specific systems or events.

Security Managers can define threshold based alert conditions on activities that may indicate attempts to gain unauthorized access and/or abuse system privileges.

Fine grained authorizations enable the Security Manager to restrict auditors and other users to information from specific sources, allowing a single repository to be deployed for an entire enterprise spanning multiple organizations.


Enterprise Audit Data Consolidation and Lifecycle Management
-----------------------------------------------------------------------------
Native audit data provides a complete view of database activity along with full execution context irrespective of whether the statement was executed directly, through dynamic SQL, or through stored procedures.

In addition to consolidating audit data from databases, operating systems, and directories, the Audit Collection Plugin can be used to collect audit data from application tables or XML files, and transfer them to the Audit Vault Server.

Audit data from databases is automatically purged after it has been moved to the Audit Vault Server.

Audit Vault Server supports data retention policies spanning days, weeks, or years on a per source basis, making it possible to meet internal or external compliance requirements.


Deployment Flexibility and Scalability
--------------------------------------------------
Security controls can be customized with in-line monitoring and blocking on some databases and monitoring only on other databases. The Database Firewall can be deployed in-line, out-of-band, or in proxy mode to work with the available network configurations.

For monitoring remote servers, the Audit Vault Agent on the database server can forward the network traffic to the Database Firewall. Delivered as a soft appliance, a single Audit Vault Server can consolidate audit logs and firewall events from thousands of databases. Both Audit Vault Server and the Database Firewall can be configured in a HA mode for fault tolerance.


These are main components of the products, for more information visit product home page.

References:
---------------
1- Oracle Product Help.
2- All images from oracle web site.

Wednesday, December 19, 2012

Database: Oracle SQLPLUS command has vague behavior with comments.

In every project I am architecting, one of the main parts of architecture is the database architecture. Database architecture consists of developing ERD, meta-data document and finally the database script.

Database script is the organized and final set of files that allow DB administration, Testing team and Developers(runs script locally on their machines) to generate final database.

DB Scripts consist of a set of files. Each file contains related database types as separate .sql file. For example; tables has a file, triggers, types, packages, user and table space, views, and lookups ….. etc all of them has separate file.

The separation makes the final script maintainable.

Finally develop an OS (Windows, Mac, UNIX and Linux) specific script file to call all of the above described files in certain order the build final database from SQLPLUS.

One of interesting vague behavior I found while running the script is something like this:





This error "ERROR at line 1: ORA-00911: invalid character" prevents anything from being created, inserted, or compiled successfully.

To not fall in error like this again, which is inserting anything after the semicolon, don't code with the following code forms in the script (It will work from JDeveloper or SQL Developer):
  1. DDL OR DML STATEMENTS ; -- any comment.
  2. DDL OR DML STATEMENTS ; REM any comment.
  3. DDL OR DML STATEMENTS ; /*any comment.*/
The Solution is to put any comments before the semicolon as the following:
Bad example:
----------------


Good example:
------------------

Saturday, December 8, 2012

Glassfish3+ & ADF 11g: Developing ADF apps on Glassfish using Jdeveloper 11g

There was no opportunity before to develop an Oracle ADF (Application Development Framework) applications on any open source application server such as Tomcat, or Glassfish, while Glassfish is the Oracle reference implementation now for JEE specifications. The only way is to develop on Oracle Weblogic application server. Even using oracle enterprise pack (OEPE) for development on eclipse IDE.

Nowadays oracle releases the ADF essentials.With the new Oracle ADF Essentials which is offering you now the ability to deploy applications that leverage Oracle ADF on the open source Glassfish 3.1 server.

At Oracle ADF Essentials home page you can check what are supported in comparison with the full ADF library, and also there are a lot of tutorials for learning the framework.

Now will get an introduction on how to setup the development environment, then we will develop a small application using Jdeveloper 11g and deploy it on the Glassfish 3.1.2.2 application server.

  • Setting up the environment:

    1. Download the following if you don’t have any of them:ADF essential library (11.1.2.3), JDeveloper 11g (11.1.2.3.0) and Glassfish 3.1.2.2.
    2. Install Jdeveloper 11g and Glassfish server and check that everything is up and running.
    3. Once ADF essential library downloaded, we need to unzip it inside the lib folder of Glassfish domain1 (or any domain you have created) this will get you an adf-essentials.zip file.

    4. To install ADF essentials:

      1. Copy the adf-essentials.zip file into the lib(on a default windows install this would be: "C:\glassfish3\glassfish\domains\domain1\lib\ext") folder of your glassfish domain.
      2. In lib folder open command window and issue a "unzip -j adf_essentials.zip"


      3. Note: you should use "-j" option with unzip (JDK program) command or you will ends up with folders not the jar files, try it and you will see.

    5. After successful extraction of the ADF libraries to the directory. Now you can start the Glassfish server (you can start your domain by the following command inside "bin" folder of your "domain" "asadmin start-domain domain1" from command line console, wait for domain to start then you should ends up with message "Command start-domain executed successfully.").

    6. We need to configure the glassfish to work with application of ADF type:
      1. Invoke the admin console of glassfish (http://localhost:4848) and log into your admin account.
      2. Go to Configurations->Server-config->JVM Settings and choose the JVM Options tab
      3. Add the following entries:
        1. -XX:MaxPermSize=512m (note this entry should already exist so just make sure it has a big enough value)
        2. -Doracle.mds.cache=simple
    7. There is a new extension ("Glassfish 1.3" as time of writing) that allows you to start and stop an external GlassFish instance, as well as start it in debug mode (which will allow JDeveloper to remotely debug your application as it runs on the server. Also a button that will invoke the web admin console of Glassfish.

    8. After installing Jdeveloper, open it, the extension is available from help->check for updates, or you can download it directly from here, and then use help->check for updates pointing to the local zip file (Thanks Shay Shmeltzer's who developed this extension).

    9. After the Jdeveloper finished download and the installation of the extension, you should find 4 glassfish buttons in the menu.


    10. Go to Tools -> Preferences -> Glassfish Preferences, here you can configure the glassfish paths and configurations you like.

  • Developing and deploying the ADF application:

    1. Creating and configuring the application to work with Glassfish:

      1. In JDeveloper choose File -> new, the New Gallery window will pop up, from right hand menu choose General-> Applications, and then from left hand choose Fusion Web Application.


      2. After successful application creation.
      3. Go into the project properties of your viewController project, under the deployment section click to edit the deployment profile that is defined there.
      4. Go to Platform and choose Glassfish 3.1 from the drop down list. Click ok to go back to your project properties.
      5. Under Java EE Application section in project properties change the Java EE web application name and context to the same value "ADFGF" (for ease of access when deploy the application).
      6. Go to Application -> Application Properties-> Deployment
      7. Go to Platform and choose Glassfish 3.1 from the drop down list. Click ok to go back to your project.
      8. This step will make sure that JDeveloper will automatically add the necessary ADF libraries to the EAR file that is being generated for deployment on Glassfish.

    2. Developing and Deploying the application:

      1. Right click on viewController project and choose new.
      2. Under Web Tier -> JSF/Facelets -> Page -> Ok.
      3. File name: TestADFGF.jsf, under the managed bean tab name: testADFGF leave the others with default values. Click ok.
      4. The page will be opened in visual editor.
      5. Drag and drop any components you like as a command button, check box, choose date and choose color component.
      6. We need to create application server connection.
      7. Start the application server from glassfish start icon in the JDeveloper menu.
      8. From Application servers Navigator, right click on application servers -> new. From New Gallery windows choose General-> Connections -> Application Server Connection -> ok.
      9. In the new windows Connection name: Glassfish3122, type: Glassfish 3.1 next.
      10. Provide your application server username and password then next.
      11. Configure your ports and host name, next.
      12. In the final step with glassfish running hit test connection button and the four testing steps should succeed.
      13. Go to your Application->Deploy and deploy either to an EAR file or directly to a Glassfish server connection that you created.
      14. After successful deployment you should get the following message:
      15. Open you browser and type the following URL (http://localhost:9090/ADFGF/faces/TestADFGF.jsf), you should see the following result:
      16. Things should just work, but if they don't then look up the server.log in the log directory and check out what error is in there.

Congratulations, you did it, now you are have the capabilities to develop any JSF/ADF applications from JDeveloper and host it on Glassfish 3.1.2.2.

Glassfish 3.1.2.2: The server exited prematurely with exit code 0

While I am issuing the following command "asadmin start-domain domain1" which yields running the glassfish 3.1.2.2 application server, after the following message "Waiting for domain1 to start ....." I have got the following error after a long time of …………….:

I was wandering what is the cause of this error; I have tried 4 times but failed with the same error, I decided to delete the old domain and create a new once, but before going to this panic way, after investigation of what is the last thing I did on my machine that might cause this error, I found that I have installed Oracle XE 11g edition which takes the same port 8080 and it was up and running.

I have stopped all the database services and tried to start the Glassfish again but this time it succeeds to run. The problem is that the glassfish port was registered by oracle database, but the error was vague and misleading somehow.

Finally I have changed my glassfish ports settings from admin console.


Friday, December 7, 2012

Database: How to uninstall Oracle database 11g Enterprise edition?

I wanted to work on database 11g XE release 1 instead of 10g for evaluating the new features, especially those ones introduced in PL/SQL, but the main problem is from 5 months I tried to install XE on windows 7 x64 bit and oracle release only x86 bit version, and when I tried to install it I have got an error indicating that a registry key file is not located and installation doesn't continue, after a while I have downloaded Oracle database 11g release 2 enterprise edition and install it, but with the time I have found that it tooks a lot of memory and disk space on my local development laptop.

Today I have downloaded and installed the express edition 11g release 2 for small foot print in memory and disk as well, the installation goes successfully this time even the error raised but this time the installation continues till the end. After successful installation of XE edition I decided to remove the enterprise one.

By the way when check uninstall program tool in windows control panel, you will not find any installation entry for enterprise edition, for xe version only.

By convenience Oracle remove its database when you run the installer again and choose to remove the installed version from there, but with 11g the story is totally different this option isn't there! Oh then how to uninstall the database?

After some search in my local oracle installation home directory I have found the solution and here is it:
  1. Navigate for your oracle home directory on the path it was installed for example "C:\Oracle\database\product\11.2.0\dbhome_1\deinstall".
  2. Inside the de-install folder right click on "deinstall.bat" run as Administrator.
  3. Wait till the tool runs and you should be asked about the version you want to de-install.
  4. Select your option and hit enter.
  5. Then you will be asked about the single listener list to be de-configured just hit enter key.
  6. Then you will be asked on database names to be also de-configured type all and hit enter.
  7. You will see message indicating that the database cleaning is in progress.
  8. After a while the tool will give you some checks and confirms that you would like to continue removing the database, type "y" and hit enter.
  9. After some minutes, all services should be removed and all major product folders will be deleted, if there are still any remaining folders, indeed you can delete it safely.
  10. Congratulation you are done.