Pages

Tuesday, November 10, 2015

How to search only in Gmail Primary tab Inbox

Provide search term and filter other categories using - operator:

search_term in:inbox -category:{social promotions updates forums}

--NO_CROSS_POST

Monday, November 9, 2015

External jars not getting picked up in zeppelin or cli in spark, issue with mysql and spark dependency

If you cant get External jars picked up in zeppelin or cli in spark , this will help you. I first tried pointing to local jars but that did not work, you can resolve using maven dependencies as you will see below In this example I am trying to use a mysql jdbc jar in zeppelin and cli and getting errors But getting an exception : java.sql.SQLException: No suitable driver found for jdbc:mysql://localhost:3306/hive at java.sql.DriverManager.getConnection(DriverManager.java:596) at java.sql.DriverManager.getConnection(DriverManager.java:187) Apparently this is not very documented feature of Spark (and not an issue with Zeppelin itself) Here is the code that works for me and solves the similar issue: Dependency loading When your code requires external library, instead of doing download/copy/restart Zeppelin, you can easily do following jobs using %dep interpreter. Load libraries recursively from Maven repository Load libraries from local filesystem Add additional maven repository Automatically add libraries to SparkCluster (You can turn off) Dep interpreter leverages scala environment. So you can write any Scala code here. Here's usages.
%dep
z.reset() // clean up previously added artifact and repository

// add maven repository
z.addRepo("RepoName").url("RepoURL")

// add maven snapshot repository
z.addRepo("RepoName").url("RepoURL").snapshot()

// add artifact from filesystem
z.load("/path/to.jar")

// add artifact from maven repository, with no dependency
z.load("groupId:artifactId:version").excludeAll()

// add artifact recursively
z.load("groupId:artifactId:version")

// add artifact recursively except comma separated GroupID:ArtifactId list
z.load("groupId:artifactId:version").exclude("groupId:artifactId,groupId:artifactId, ...")

// exclude with pattern
z.load("groupId:artifactId:version").exclude(*)
z.load("groupId:artifactId:version").exclude("groupId:artifactId:*")
z.load("groupId:artifactId:version").exclude("groupId:*")

// local() skips adding artifact to spark clusters (skipping sc.addJar())
z.load("groupId:artifactId:version").local()
Note that %dep interpreter should be used before %spark, %pyspark, %sql. Thanks to Ali and Neeraj from HWX for help in solving this issue.

Thursday, February 26, 2009

Setting Up A New Web Application Using Eclipse, Maven, Ant and Tomcat

Create a new application structure using maven

mvn archetype:create -DgroupId=com.zeltov.appstore -DartifactId=appstore2 -DarchetypeArtifactId=maven-archetype-webapp

Create Eclipse project

mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true

In Eclipse import as a maven project

Modify the pom.xml file:

<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>${spring-version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>${spring-version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>taglibs</groupId>
<artifactId>standard</artifactId>
<version>${taglibs-standard-version}</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>${jstl-version}</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>${servlet-api-version}</version>
<scope>provided</scope>
</dependency>
</dependencies>
<properties>

<spring-version>2.5.6</spring-version>
<junit-version>4.1</junit-version>
<jstl-version>1.2</jstl-version>
<taglibs-standard-version>1.1.2</taglibs-standard-version>
<servlet-api-version>2.5</servlet-api-version>
</properties>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>info.jtrac</groupId>
<artifactId>maven-antprops-plugin</artifactId>
</plugin>
</plugins>
</build>
<pluginRepositories>
<pluginRepository>
<id>jtrac.info</id>
<url>http://j-trac.sourceforge.net/repository</url>
</pluginRepository>
</pluginRepositories>


Next Run an ant command to generate the jar dependencies property file


mvn antprops:generate


To download mvn dependencies with sources:

mvn dependency:sources


Maven Ant Integration Tutorial by Peter Thomas great way to do dependency management and use ant at the same time

Article to create a java app and then a webapp as 2 seperate projects using maven