HBase SQL statement fails with Insufficient permissions for user

ISSUE: Below error comes up when creating new hbase table hbase(main):001:0> create 'anoop','cf1' ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'anoop' (global, action=CREATE) at org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:426) at org.apache.hadoop.hbase.security.access.AccessController.preCreateTable(AccessController.java:563) Solution: - Once the cluster has been secured, a user has to authenticate itself to kerberos by doing a kinit. By default, hbase is a superuser who was … Continue reading HBase SQL statement fails with Insufficient permissions for user


HBASE snapshots How To

What is a Snapshot? A snapshot is a set of metadata information that allows an admin to get back to a previous state of the table. A snapshot is not a copy of the table; it’s just a list of file names and doesn’t copy the data. A full snapshot restore means that you get … Continue reading HBASE snapshots How To

Steps to use security enabled kafka

Steps to use kerberos security enabled kafka are below.   Set Inter Broker Protocol SASL_PLAINTEXT in Cloudera manager Create a jaas.conf file with the following contents to use with cached Kerberos credentials For kinit KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true; }; For keytab   KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/username.keytab" principal="username@realm"; }; In the above jaas.conf file the user's … Continue reading Steps to use security enabled kafka


Spark2, PySpark and Jupyter installation and configuration

Steps to be followed for enabling SPARK 2, pysaprk and jupyter in cloudera clusters. 1.INSTALL ORACLE JDK IN ALL NODES Download and install java. It should be jdk 1.8+ # cd /usr/java/ # wget –no-cookies –no-check-certificate –header “Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie” “http://download.oracle.com/otn-pub/java/jdk/8u144-b15/jdk-8u144-linux-x64.tar.gz” # tar xzf jdk-8u144-linux-x64.tar.gz   2.Install java with Alternatives # cd /usr/java # alternatives … Continue reading Spark2, PySpark and Jupyter installation and configuration


Creating multiple spark sessions in kerberos enabled cluster throws error

ISSUE:Creating multiple spark sessions in kerberos enabled cluster throws below error Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. : org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:7519) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getDelegationToken(NameNodeRpcServer.java:548) Solution: Use the keytab and principal wihin spark code as below   spark = SparkSession\     .builder\     .appName('asdf')\     … Continue reading Creating multiple spark sessions in kerberos enabled cluster throws error


PostgreSQL basics

To login to prompt #psql -h myhost -d mydb -U myuser -W To create user psql#CREATE USER anoop WITH password 'tmppassword'; To create database  CREATE DATABASE db1 WITH OWNER anoop; List databases \l List all the tables \d Turn on timing \timing      


How to generate kerberos keytabs

For AD accounts 1.From shell #ktutil 2. Add an entry as below ktutil:addent -password  -p username@realm.com -k 1 -e RC4-HMAC ktutil:wkt  username.keytab ktutil:q For FeeIPA accounts Use the below command #ipa-getkeytab -s ipa.host.com -p username@realm.com  --keytab=username.keytab --password Try kinit using the keytab as below #kinit username@realm.com -k -t username.keytab Note: Change the username  accordingly.