GSoC 2016 - Apache Zeppelin

Questions

To debug I add the following line in zeppelin-env.sh
```
export ZEPPELIN_JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9009"
```
I use Intellij Idea IDE and then remote debug.I set the breakpoints and able to watch all the variables. when I run the paragraph I get Socketexception as paragraph output, after taking too much time to run but commenting the debug line in zeppelin-env.sh I don't get the error ?
The dependency structure of project is as follows
zeppelin-server <= zeppelin-zengine <= zeppelin-interpreter
So do I have to always package the root project or just the module in which I made change, followed by zeppelin root package ?
```
<dependency>
      <groupId>${project.groupId}</groupId>
      <artifactId>zeppelin-interpreter</artifactId>
      <version>${project.version}</version>
</dependency>
```
Where does it look for dependency in target folder of zeppelin-interpreter or .m2 local repository. ?

Xml regarding issues

The config member of Paragraph class is Map<String,Object> or to be precise HashMap<String,Object> following the flow this is how it is set..

In NotebookServer the OnMeassagereceived() function has a switch case based on the operation. The updateParagraph and runParagraph methods take the config value from Message.data and also other values etc. like params. The runtime values of config values is com.google.gson.internal.StringMap. The values can itslef be Float, Integer, Boolean , Arraylist<StringMap>. So the issue is only the root level entries are mapped in Xml and the graph element of config is empty in Xml.

Second issue is mapping back from Xml to Note. The error I am getting is InvocationTargetException not Jaxbexception just after unmarshalling. So I was not able to properly debug line

Hence to work around this I delete the notes in notebook-xml and re run so the notebookreposync sync(0,1) converts to xml back for me.

Goal

The goal would be to have .xml representation of the notebook persisted in local filesystem along with existing .json one. Could be just note.xml in the same folder, or could be `./notebook-xml/<noteId>/note.xml`
It should save the same notebook, but in XML format, just in the local filesystem.

So here is how I approached..

Created XmlNotebookRepo java class in
package
org.zeppelin.notebook.repo; copied code from VFSNotebookRepo changed the storage directory at this line
```
this.filesystemRoot = new URI(new File(
        conf.getRelativeDir(filesystemRoot.getPath() + "-xml")).getAbsolutePath()); 
```

Added the following in zeppelin-site.xml

<property>
        <name>zeppelin.notebook.storage</name>
        <value>org.apache.zeppelin.notebook.repo.XmlNotebookRepo</value>
        <description>notebook persistence layer implementation</description>
</property>

So now zeppelin.notebook.storage would have two properties but while remote debugging I found that it has only one value. I also tried uncommenting the GitNotebookRepo storage property but still the value was one.

the allStorageClassNames variable did not contain comma separated class names.

So I proceeded with the XmlNotebookRepo itself.

JAXB Usage

I read and created some JAXB examples created some Employee, Student, Address examples with composition.Generated the XML output, different types of annotations @XmlRootElement , @XmlElementWrapper , @XmlElement , @XmlAccessorType(XmlAccessType.FIELD) @XmlTransient etc. This blog[1] was quite useful.

A no-arg constructor is required.
Public getter/setter or @XmlElement
Also java collection are mapped to Xml like Map,List,Set. @XmlElementWrapper to create a wrapping element.

Mapping of Interfaces.

Caused by: com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 2 counts of IllegalAnnotationExceptions java.util.List is an interface, and JAXB can't handle interfaces

The solution to this is to use @XmlAnyElement with @XmlRootElement and passing .class of the classes that implement it to JAXBContext.newInstance. Here is the code to demonstrate. Here are the following classes[2]

Address.java
Cow.java
Employee.java
Student.java
XMLTest.java

Now moving to zeppelin, Modifying Note.java with the annotations, also Paragraph.java needs to modified but I thought lets start first with simple fields to mapped and later move to complex members like AngularObjects, paragraphs, config, Info.

Fields like NoteInterpreterLoader, JobListenerFactory , NotebookRepo have to be transient , we don't want them to be mapped in XML file so I used the annotation @XmlTransient and still I am getting error at this line in XmlNotebookRepo save() method.

Please help.. I have spent a day solving this I don't know how to proceed further.
Before running the server again please delete the note in notebook-xml as I have not handled loadallNotes() which loads all saved notes. Breakpoint is at save method in XmlNotebookrepo which is hit after creating Note in the UI.

Errors are temporary, giving up is permanent

So I have figured out what was wrong. I read about XmlAccessorTypes like Field,Property and Public and also XmlAdapters. Now I have my Note saved in Xml partially AngularObjects and GUI-config is remaining and loading of notes i.e unmarshalling.

note.xml