Showing posts with label mariadb. Show all posts
Showing posts with label mariadb. Show all posts

Thursday, 5 January 2023

Connecting remotely to a local MySQL Server

So I tried it, and it works. Here's some notes.

ssh -N -L 3306:127.0.0.1:3306 mrbear@bears.jelastic.eapps.com -p 3022
mysql -u root -p mrbeardb —port=3306

References

Linuxize - How to Connect to MySQL through SSH Tunnel
https://linuxize.com/post/mysql-ssh-tunnel/

Thursday, 18 November 2021

Running Mariadb on the Raspberry Pi

This is going to be a small blurb, on what I did to get this working.

Nothing out of the ordinary, really.

$ apt-get install mariadb-server
$ cd /etc/mysql/mariadb.conf.d
$ joe 50-server.cnf
# started vi and replaced 127.0.0.1 with 0.0.0.0
$ service mysql restart

And of course allow the user accounts to access mariadb remotely over the network.

Attention! I'm only doing this locally on a testing server! It's not a great idea to run mariadb connections remotely if you do not have to.

GRANT ALL PRIVILEGES ON *.* TO 'root'@'192.168.100.%' IDENTIFIED BY 'my-new-password' WITH GRANT OPTION;

Friday, 29 May 2020

Adding MariaDB JDBC Connector to Paraya 5 - Followup

Just decided to do a followup on Adding MariaDB JDBC Connector to Payara 51.

I wanted to see if it was possibly very easy to do this using the command line.

Installing a MariaDB driver

$ ./asadmin add-library /home/mrbear/software/mariadb-java-client-2.3.0.jar
Command add-library executed successfully.

Creating a new JDBC Connection pool

The syntax:

Usage: asadmin [asadmin-utility-options] create-jdbc-connection-pool
        [--datasourceclassname ] [--restype ]
        [--steadypoolsize ]
        [--maxpoolsize ]
        [--maxwait ]
        [--poolresize ]
        [--idletimeout ] [--initsql ]
        [--isolationlevel ]
        [--isisolationguaranteed[=]]
        [--isconnectvalidatereq[=]]
        [--validationmethod ]
        [--validationtable ]
        [--failconnection[=]]
        [--allownoncomponentcallers[=]]
        [--nontransactionalconnections[=]]
        [--validateatmostonceperiod ]
        [--leaktimeout ]
        [--leakreclaim[=]]
        [--creationretryattempts ]
        [--creationretryinterval ]
        [--sqltracelisteners ]
        [--statementtimeout ]
        [--statementleaktimeout ]
        [--statementleakreclaim[=]]
        [--lazyconnectionenlistment[=]]
        [--lazyconnectionassociation[=]]
        [--associatewiththread[=]]
        [--driverclassname ]
        [--matchconnections[=]]
        [--maxconnectionusagecount ]
        [--ping[=]] [--pooling[=]]
        [--statementcachesize ]
        [--validationclassname ]
        [--wrapjdbcobjects[=]]
        [--description ] [--property ]
        [-?|--help[=]] jdbc_connection_pool_id

Note the following:

  • The escape character backslash (\) is used in the --property option to distinguish the semicolon (;).
  • Two backslashes (\\) are used to distinguish the equal sign (=).
  • Two backslashes (\\) are used to distinguish the colon (:).
./asadmin create-jdbc-connection-pool --driverclassname org.mariadb.jdbc.Driver --restype java.sql.Driver --property databaseName=mmud:password=topsecretpassword:user=root:URL=jdbc\\:mariadb\\://localhost\\:3306/mmud:serverName=localhost mmudpool

Let's test it!

./asadmin ping-connection-pool mmudpool

In case you wish to delete it, try:

./asadmin delete-jdbc-connection-pool mmudpool

Creating the JNDI

./asadmin create-jdbc-resource --connectionpoolid mmudpool jdbc/mmud

References

[1] Adding MariaDB JDBC Connector to Payara 5

https://randomthoughtsonjavaprogramming.blogspot.com/2018/12/adding-mariadb-jdbc-connector-to-paraya.html
[2] Payara Blog - An intro to connection pools in payar server 5
https://blog.payara.fish/an-intro-to-connection-pools-in-payara-server-5

Tuesday, 14 April 2020

Backup & Restore of Mariadb Server

Hello, again!

This is just a small blog post on how to backup and restore a MariaDB server, without using MysqlDump, but by directly copying the files in /var/lib/mysql.

Backing up

First of course we need to shutdown mariadb.

# systemctl status mariadb
# systemctl stop mariadb

Copying files:

tar zcvf database_backup_`date +%F`.tgz /var/lib/mysql

Restoring

When putting things back, things start to go wrong1.

Copying the files to a new host, causes this:

2020-04-14 10:45:32 0 [ERROR] InnoDB: Operating system error number 13 in a file operation. 2020-04-14 10:45:32 0 [ERROR] InnoDB: The error means mysqld does not have the access rights to the directory. 2020-04-14 10:45:32 0 [ERROR] InnoDB: os_file_get_status() failed on './ibdata1'. Can't determine file permissions

So, if the below code works, than SELinux is causing the issue:

sudo semanage permissive -a mysqld_t

Now you added mysql_t to permissive, as shown by the below printout:

]# semanage permissive -l

Builtin Permissive Types


Customized Permissive Types

mysqld_t

Turning it back to enforcing, means running:

# semanage permissive -d mysqld_t
libsemanage.semanage_direct_remove_key: Removing last permissive_mysqld_t module (no other permissive_mysqld_t module exists at another priority).

We need to relabel the new /var/lib/mysql directory:

sudo semanage fcontext -a -t mysqld_db_t "/var/lib/mysql(/.*)?"
sudo restorecon -Rv /var/lib/mysql

Checking the current file contexts:

$ ls --directory --context /var/lib/mysql
unconfined_u:object_r:mysqld_db_t:s0 /var/lib/mysql

MariaBackup

I need to check out MariaBackup2 3, as a better solution for creating backups compared to mysqldump.

References

[1] Security-Enhanced Linux with MariaDB
https://mariadb.com/kb/en/selinux/
[2] Backing up and restoring databases
https://mariadb.com/kb/en/backing-up-and-restoring-databases/
[3] MariaBackup
https://mariadb.com/kb/en/mariabackup/
[4] Full Backup and Restore with MariaBackup
https://mariadb.com/kb/en/full-backup-and-restore-with-mariabackup/

Thursday, 30 January 2020

MariaDB issues

I ran into some issues, and I thought I'd document them here.

Not able to detect platform for vendor name [MariaDB1010.3.17-MariaDB]. Defaulting to [org.eclipse.persistence.platform.database.DatabasePlatform]. The database dialect used may not match with the database you are using. Please explicitly provide a platform using property "eclipselink.target-database".]]

So changed my persistence.xml and added:

<property name="eclipselink.target-database" value="MySQL4"/>

Also:

Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.7.4.payara-p2): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: (conn=16) Table 'mmud.SEQUENCE' doesn't exist
Error Code: 1146
Call: UPDATE SEQUENCE SET SEQ_COUNT = SEQ_COUNT + ? WHERE SEQ_NAME = ?
       bind => [2 parameters bound]
Query: DataModifyQuery(name="SEQ_GEN_IDENTITY" sql="UPDATE SEQUENCE SET SEQ_COUNT = SEQ_COUNT + ? WHERE SEQ_NAME = ?")
       at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)

As the vendor name was not detected, eclipselink switched to a default implementation. The default implementation requires SEQUENCES for the IDENTITY definition.

It turns out the MariaDB implementation of Sequences is not compatible with the standard SQL way of creating sequences.

So I had to add the following property to the JDBC Connection pool:

useMysqlMetadata = true

References

JIRA MariaDB : since 2.4.0 j-connector throws sequence errors via JPA/ eclipselink on @GeneratedValue(strategy = GenerationType.IDENTITY) columns
https://jira.mariadb.org/browse/CONJ-702
Java Persistence API (JPA) Extensions Reference for EclipseLink, Release 2.4 : target-database
https://www.eclipse.org/eclipselink/documentation/2.4/jpa/extensions/p_target_database.htm
Eclipse JIRA : Bug 462196 - Add support for MariaDB
https://bugs.eclipse.org/bugs/show_bug.cgi?id=462196

Thursday, 23 January 2020

Using SQL to generate JSON output

I recently read [1], and it had a very interesting notion.

The idea is to let the database generate JSON, and provide it straight into your client.

So I decided to find out if MariaDB had some support for this as well.

It does2.

So, it basically was nothing more then calling a NativeQuery on the EntityManager, and returning a concatted resultset with a '['prefix and a ']'postfix and a comma-delimiter and away we go.

The native query looked like the following:

It worked flawlessly!

Caveat: of course, this is only in the case where your middleware (as in this example) really doesn't need to do anything with the result.

I mean, you are going to have to do all checks in the database query.

I think this example shows its strength when you just really want to read a lot of data, and do not need to process it.

References

[1] Stop Mapping Stuff in Your Middleware. Use SQL’s XML or JSON Operators Instead
https://blog.jooq.org/2019/11/13/stop-mapping-stuff-in-your-middleware-use-sqls-xml-or-json-operators-instead/
[2] MariaDB - starting with 10.2.3 - JSON Functions
https://mariadb.com/kb/en/library/json-functions/
MariaDB - JSON with MariaDB Platform: What Is JSON and Why Use It – With Examples
https://mariadb.com/resources/blog/json-with-mariadb-10-2/
MariaDB - Relational and Semi-structured Data
https://mariadb.com/database-topics/semi-structured-data/
MariaDB - Function Differences Between MariaDB 10.4 and MySQL 8.0
https://mariadb.com/kb/en/library/function-differences-between-mariadb-104-and-mysql-80/#present-in-mysql-only

Thursday, 28 February 2019

Installing MariaDB 10 on CentOS 7

I have always used the stock install of MariaDB that came with CentOS 7, but I noticed recently that it is still only on 5.5 (at the time of writing, 5.5.60).

I never had any issues with it, until I tried to upgrade my Liferay CMS system. Apparently there are definite changes that are not incorporated into the 5.5.

I wished to moved to 10.3, which is the latest stable branch, and I can apparently do that (as indicated on the mariadb website1) by creating a MariaDB repository for DNF and then installing as usual.

Of course, first uninstall the 5.5 version.

References

[1] MariaDB - Installing MariaDB 10 on CentOS 7 / RHEL 7
https://mariadb.com/resources/blog/installing-mariadb-10-on-centos-7-rhel-7/
MariaDB - Moving to MariaDB Backup
https://mariadb.com/resources/blog/moving-to-mariadb-backup/

Thursday, 27 December 2018

Adding MariaDB JDBC Connector to Paraya 5

It's as easy as pie.

  1. start Payara
    $ cd ~/payara5/bin
    $ ./asadmin start-domain
    Waiting for domain1 to start ...............
    Successfully started the domain : domain1
    domain Location: /home/mrbear/payara5/glassfish/domains/domain1
    Log File: /home/mrbear/payara5/glassfish/domains/domain1/logs/server.log
    Admin Port: 4848
    Command start-domain executed successfully.
  2. install the database driver
    $ ./asadmin add-library /home/mrbear/software/mariadb-java-client-2.3.0.jar
    Command add-library executed successfully.
    or try:
    $ cp ~/Downloads/mariadb-java-client-2.3.0.jar ~/payara5/glassfish/domains/domain1/lib

    The last part will probably require a payara application server reboot, though.

  3. connect to http://localhost:4848
  4. surf to Resources -> JDBC -> JDBC Connection Pools
  5. Create a new one
  6. Settings:
    Pool name[obvious]
    Resource Typejavax.sql.DataSource
    Datasource classnameorg.mariadb.jdbc.MariaDbDataSource

    Leave the rest as default.

  7. Define the following additional properties:
    NameValue
    serverNamelocalhost
    URLjdbc:mariadb://localhost:3306/mmud
    databaseNamemmud
    userusername
    passworditsasecret
  8. Try the "Ping" button to verify that the connection is established.
  9. surf to Resources -> JDBC -> JDBC Resources
  10. Create a new one
  11. Settings:
    NameValue
    JNDI Namethis is the name as the resource is defined in your application
    Pool Namethe pool defined previously, it's a dropdown which is convenient.
  12. Press finish or OK or something
  13. Done!

References

Using MySQL with Payara Server
https://blog.payara.fish/using-mysql-with-payara
Payara - Using Hibernate 5 on Payara Server
https://blog.payara.fish/using-hibernate-5-on-payara-server

Thursday, 6 December 2018

Adding MariaDB JDBC Connector to Wildfly 14

  1. install the database driver as a module
    1. create directory WILDFLY_HOME/modules/com/mariadb/main
      $ mkdir -p modules/com/mariadb/main
    2. copy database driver into that one
      $ cp ~/Downloads/mariadb-java-client-2.3.0.jar .
    3. Create the file module.xml in the same folder with the following content:
      <module xmlns="urn:jboss:module:1.3" name="com.mariadb">
          <resources>
              <resource-root path="mariadb-java-client-2.3.0.jar"/>
          </resources>
          <dependencies>
              <module name="javax.api"/>
              <module name="javax.transaction.api"/>
          </dependencies>
      </module>
  2. start Wildfly
    $ cd $WILDFLY_HOME/bin
    $ ./standalone.sh
  3. connect to http://localhost:9990/error/index.html
  4. run ./add-user.sh
    [mrbear@localhost bin]$ ./add-user.sh 

    What type of user do you wish to add? 
     a) Management User (mgmt-users.properties) 
     b) Application User (application-users.properties)
    (a): a

    Enter the details of the new user to add.
    Using realm 'ManagementRealm' as discovered from the existing property files.
    Username : mrbear
    Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
     - The password should be different from the username
     - The password should not be one of the following restricted values {root, admin, administrator}
     - The password should contain at least 8 characters, 1 alphabetic character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
    Password : 
    Re-enter Password :  
    What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]: 
    About to add user 'mrbear' for realm 'ManagementRealm'
    Is this correct yes/no? yes
    Added user 'mrbear' to file '/home/mrbear/wildfly-14.0.1.Final/standalone/configuration/mgmt-users.properties'
    Added user 'mrbear' to file '/home/mrbear/wildfly-14.0.1.Final/domain/configuration/mgmt-users.properties'
    Added user 'mrbear' with groups  to file '/home/mrbear/wildfly-14.0.1.Final/standalone/configuration/mgmt-groups.properties'
    Added user 'mrbear' with groups  to file '/home/mrbear/wildfly-14.0.1.Final/domain/configuration/mgmt-groups.properties'
    Is this new user going to be used for one AS process to connect to another AS process? 
    e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
    yes/no? no
  5. click on "Try again" on the Website
  6. log on with the new added user
  7. Configuration - Subsystems - Datasources and drivers - JDBC drivers - [+]
  8. Fill out the following information:
    Driver namemariadb
    Driver Module Namecom.mariadb
    Module Slot
    Driver Class Nameorg.mariadb.jdbc.Driver
    Drive Datasource Class Name
    Driver XA Datasource Class Name
  9. Configuration - Subsystems - Datasources and drivers - Datasources - [+]
  10. Select "Add Datasource" to add a non-XA Datasource
  11. Select "Custom" template
  12. Attributes:
    Name[name]
    JNDI Namejava:/jdbc/[name]
  13. JDBC Driver
    Driver Namemariadb
    Driver Module Name[empty]
    Driver Class Name[empty]
  14. Connection:
    Connection URLjdbc:mariadb://localhost:3306/[databasename]
    User nameroot
    Passworditsasecret
    Security Domain[empty]
  15. Test connection
  16. Reload of server required - so reload the server
  17. Test connection: Successfully tested connection for datasource [name].

References

JDBC Driver / DataSource Setup
https://hal.gitbooks.io/dev/content/recipes/jdbc-driver-setup.html
About MariaDB Connector/J
https://mariadb.com/kb/en/library/about-mariadb-connector-j/
GitHub - Use MariaDB Connector/J Driver
https://github.com/MariaDB/mariadb-connector-j/blob/master/documentation/use-mariadb-connector-j-driver.creole
IronJacamar 1.1 - Userguide - Deploying DS Descriptor
http://www.ironjacamar.org/doc/userguide/1.1/en-US/html_single/index.html#deployingds_descriptor

Saturday, 2 December 2017

Automated executing of MySQL/MariaDB scripts

I am running MariaDB and I wish to execute sql scripts without all this hassle of entering my password. Of course this carries severe security risks with it, that we need to be aware of and, if possible, mitigate.

Via the commandline

It is possible to execute sql scripts via the commandline1, but the problem here is that the password you use is visible in the process list. So this is a security risk.

Let's not do this.

mysql_config_editor

I firstly checked out mysql_config_editor2, which enables you to put the password and other options into an encrypted configuration file. But it turns out that MariaDB does not come with that specific tool. The encryption used seems quite weak, and there's an article about the security issues at [3]. There is also the blogpost at [4] giving some details.

So now what?

Well, there is always the plan to use the configuration file .my.cnf5, and you can store your mysql or mariadb password in there and everything would be hunky-dory.

The .my.cnf looks like this:

[client]
password=topsecretpassword

You are no doubt aware that the password is stored in cleartext.

The following security measures should be in place:

  • always make sure the permissions on the file are set to -rw-------
  • create a user in your database with only those permissions that are required by your scripts. In most cases, this is select/update/delete/insert statements.
  • when you are finished with your scripts, it might be a good idea to remove the password from the conf file. I understand that with cron jobs and batch scripts this might not be possible.

There is an example of a my.cnf containing every possible configuration option at /usr/share/mysql/my-large.cnf when you install MariaDB.

References

[1] StackOverflow - How to execute a MySQL command from a shell script?
https://stackoverflow.com/questions/8055694/how-to-execute-a-mysql-command-from-a-shell-script
[2] MysqlManual 5.7 - mysql_config_editor
https://dev.mysql.com/doc/refman/5.7/en/mysql-config-editor.html
[3] MariaDb Blog -
https://mariadb.com/resources/blog/mysql-56-security-through-complacency
[4] Todd's MySQL Blog - Understanding mysql_config_editor’s security aspects
http://mysqlblog.fivefarmers.com/2012/08/16/understanding-mysql_config_editors-security-aspects/
[5] MariaDB - Configuring MariaDB with my.cnf
https://mariadb.com/kb/en/library/configuring-mariadb-with-mycnf/