GSoC 2016 - Apache Zeppelin: Week 1 - XmlNotebookRepo (Store Notebooks in XML format)

Goal

The goal would be to have .xml representation of the notebook persisted in local filesystem along with existing .json one. Could be just note.xml in the same folder, or could be `./notebook-xml/<noteId>/note.xml`
It should save the same notebook, but in XML format, just in the local filesystem.

So here is how I approached..

Created XmlNotebookRepo java class in
package
org.zeppelin.notebook.repo; copied code from VFSNotebookRepo changed the storage directory at this line
```
this.filesystemRoot = new URI(new File(
        conf.getRelativeDir(filesystemRoot.getPath() + "-xml")).getAbsolutePath()); 
```

Added the following in zeppelin-site.xml

<property>
        <name>zeppelin.notebook.storage</name>
        <value>org.apache.zeppelin.notebook.repo.XmlNotebookRepo</value>
        <description>notebook persistence layer implementation</description>
</property>

So now zeppelin.notebook.storage would have two properties but while remote debugging I found that it has only one value. I also tried uncommenting the GitNotebookRepo storage property but still the value was one.

the allStorageClassNames variable did not contain comma separated class names.

So I proceeded with the XmlNotebookRepo itself.

JAXB Usage

I read and created some JAXB examples created some Employee, Student, Address examples with composition.Generated the XML output, different types of annotations @XmlRootElement , @XmlElementWrapper , @XmlElement , @XmlAccessorType(XmlAccessType.FIELD) @XmlTransient etc. This blog[1] was quite useful.

A no-arg constructor is required.
Public getter/setter or @XmlElement
Also java collection are mapped to Xml like Map,List,Set. @XmlElementWrapper to create a wrapping element.

Mapping of Interfaces.

Caused by: com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 2 counts of IllegalAnnotationExceptions java.util.List is an interface, and JAXB can't handle interfaces

The solution to this is to use @XmlAnyElement with @XmlRootElement and passing .class of the classes that implement it to JAXBContext.newInstance. Here is the code to demonstrate. Here are the following classes[2]

Address.java
Cow.java
Employee.java
Student.java
XMLTest.java

	package com.employee.three;

	import javax.xml.bind.annotation.XmlAccessType;
	import javax.xml.bind.annotation.XmlAccessorType;

	/**
	* Created by onkar on 28/4/16.
	*/

	@XmlAccessorType(XmlAccessType.FIELD)
	public class Address {
	String city;
	String street;
	int pincode;

	public Address(String city, String street, int pincode) {
	this.city = city;
	this.street = street;
	this.pincode = pincode;
	}
	public Address(){}

	@Override
	public boolean equals(Object o) {
	if (this == o) return true;
	if (!(o instanceof Address)) return false;

	Address address = (Address) o;

	if (pincode != address.pincode) return false;
	if (!city.equals(address.city)) return false;
	return street != null ? street.equals(address.street) : address.street == null;

	}

	@Override
	public int hashCode() {
	int result = city.hashCode();
	result = 31 * result + (street != null ? street.hashCode() : 0);
	result = 31 * result + pincode;
	return result;
	}
	}

view raw Address.java hosted with ❤ by GitHub

	package com.employee.three;

	import javax.xml.bind.annotation.XmlRootElement;

	/**
	* Created by onkar on 28/4/16.
	*/

	@XmlRootElement
	public class Cow implements Animal {
	private int milkCapacity;
	private String breed;

	private Cow() {
	}

	public Cow(int milkCapacity, String breed) {
	this.milkCapacity = milkCapacity;
	this.breed = breed;
	}

	public int getMilkCapacity() {
	return milkCapacity;
	}

	public void setMilkCapacity(int milkCapacity) {
	this.milkCapacity = milkCapacity;
	}

	public String getBreed() {
	return breed;
	}

	public void setBreed(String breed) {
	this.breed = breed;
	}

	public void makeNoise() {

	}
	}

view raw Cow.java hosted with ❤ by GitHub

	package com.employee.three;

	import javax.xml.bind.annotation.XmlRootElement;

	/**
	* Created by onkar on 28/4/16.
	*/
	@XmlRootElement
	public class Dog implements Animal{
	private String name;
	private int age;
	private int noseStrength;
	private String breed;

	private Dog() {
	}

	public Dog(String name, int noseStrength, String breed) {
	this.name = name;
	this.noseStrength = noseStrength;
	this.breed = breed;
	}

	public String getName() {
	return name;
	}

	public void setName(String name) {
	this.name = name;
	}

	public int getAge() {
	return age;
	}

	public void setAge(int age) {
	this.age = age;
	}

	public int getNoseStrength() {
	return noseStrength;
	}

	public void setNoseStrength(int noseStrength) {
	this.noseStrength = noseStrength;
	}

	public String getBreed() {
	return breed;
	}

	public void setBreed(String breed) {
	this.breed = breed;
	}

	public void makeNoise() {

	}
	}

view raw Dog.java hosted with ❤ by GitHub

	package com.employee.three;

	import javax.xml.bind.annotation.XmlElement;
	import javax.xml.bind.annotation.XmlElementWrapper;
	import javax.xml.bind.annotation.XmlRootElement;
	import java.util.List;
	import java.util.Set;

	/**
	* Created by onkar on 29/4/16.
	*/

	@XmlRootElement(name="Person")
	class Employee implements Person{
	@XmlElement
	private String name;
	@XmlElement
	private long phoneNumber;
	@XmlElement
	private boolean married;
	@XmlElementWrapper
	@XmlElement
	private Set<Address> addressSet;

	private Employee(){

	}

	public Employee(String name,long phoneNumber,boolean marriedStatus,Set<Address> addressList){
	this.name=name;
	this.phoneNumber=phoneNumber;
	this.married=marriedStatus;
	this.addressSet=addressList;
	}

	public void calculateAge() {

	}
	}

view raw Employee.java hosted with ❤ by GitHub

	package com.employee.three;

	import javax.xml.bind.annotation.*;
	import java.beans.Transient;
	import java.util.List;
	import java.util.Map;
	import java.util.Set;

	/**
	* Created by onkar on 28/4/16.
	*/
	@XmlRootElement(name="Student")
	public class Student implements Person {
	@XmlElement
	private String name;
	@XmlElement
	private long Id;
	@XmlElementWrapper
	@XmlElement
	private Map<Integer,Address> addressMap;

	@XmlElementWrapper
	@XmlAnyElement
	private List<Animal> animals;


	private Student(){

	}

	public Student(String name, long id, Map<Integer, Address> addressMap,List<Animal> animals) {
	this.name = name;
	Id = id;
	this.addressMap = addressMap;
	this.animals=animals;
	}

	public void calculateAge() {

	}
	}

	/*
	Discovered this
	@XmlTransient
	on a private transient variable
	will give you error cannot handle interfaces
	better to give on getters and setters than variables
	*/

view raw Student.java hosted with ❤ by GitHub

	package com.employee.three;

	import javax.xml.bind.JAXBContext;
	import javax.xml.bind.JAXBException;
	import javax.xml.bind.Marshaller;
	import java.util.*;

	/**
	* Created by onkar on 28/4/16.
	*/

	interface Animal {
	public void makeNoise();
	}

	interface Person {
	public void calculateAge();
	}

	public class XMLTest {

	public static void main(String[] args) {

	Address a = new Address("Pune", "21 street", 521311);
	Address b = new Address("Hadapsar", "22 jump street", 12343);
	Address c = new Address("kanpur", "21 street", 8218751);
	/* Set<Address> addresses = new HashSet<Address>();
	addresses.add(a);
	addresses.add(b);
	Employee emp1 = new Employee("ramesh", 9811133, true, addresses);
	addresses.add(c);
	Employee emp2 = new Employee("sunny", 7271144, false, addresses);*/

	Map<Integer,Address> addressBook = new HashMap<Integer, Address>();
	addressBook.put(1,a);
	addressBook.put(2,b);
	addressBook.put(3,c);

	List<Animal> animals = new ArrayList<Animal>();
	animals.add(new Dog("Bruno",2,"Lab"));
	animals.add(new Cow(10,"Jersey"));

	Student emp1 = new Student("ramesh",21290,addressBook,animals);
	Student emp2 = new Student("sunny",43182,addressBook,animals);


	JAXBContext contextObj = null;
	try {
	contextObj = JAXBContext.newInstance(Student.class,Cow.class,Dog.class);
	Marshaller marshallerObj = contextObj.createMarshaller();
	marshallerObj.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
	marshallerObj.marshal(emp1, System.out);
	System.out.println();
	marshallerObj.marshal(emp2, System.out);

	} catch (JAXBException e) {
	e.printStackTrace();
	}
	}
	}

	/*
	no arg constructor is required.
	if you use xmlanyelement xmlrootelement has to be declared on class which implements interface
	both implementations '.class' must be passed in JAXBcontext.newInstance()
	@XmlElements({
	@XmlElement(type = MyImpl.class),
	@XmlElement(type = AnotherImpl.class)
	}) this should also work but the element name in XML will be interface name - animals and not cow,dog.
	*/

view raw XMLTest.java hosted with ❤ by GitHub

	?xml version="1.0" encoding="UTF-8" standalone="yes"?>
	<Student>
	<name>ramesh</name>
	<Id>21290</Id>
	<addressMap>
	<entry>
	<key>1</key>
	<value>
	<city>Pune</city>
	<street>21 street</street>
	<pincode>521311</pincode>
	</value>
	</entry>
	<entry>
	<key>2</key>
	<value>
	<city>Hadapsar</city>
	<street>22 jump street</street>
	<pincode>12343</pincode>
	</value>
	</entry>
	<entry>
	<key>3</key>
	<value>
	<city>kanpur</city>
	<street>21 street</street>
	<pincode>8218751</pincode>
	</value>
	</entry>
	</addressMap>
	<animals>
	<dog>
	<age>0</age>
	<breed>Lab</breed>
	<name>Bruno</name>
	<noseStrength>2</noseStrength>
	</dog>
	<cow>
	<breed>Jersey</breed>
	<milkCapacity>10</milkCapacity>
	</cow>
	</animals>
	</Student>

view raw zoutput.xml hosted with ❤ by GitHub

Now moving to zeppelin, Modifying Note.java with the annotations, also Paragraph.java needs to modified but I thought lets start first with simple fields to mapped and later move to complex members like AngularObjects, paragraphs, config, Info.

Fields like NoteInterpreterLoader, JobListenerFactory , NotebookRepo have to be transient , we don't want them to be mapped in XML file so I used the annotation @XmlTransient and still I am getting error at this line in XmlNotebookRepo save() method.

Please help.. I have spent a day solving this I don't know how to proceed further.
Before running the server again please delete the note in notebook-xml as I have not handled loadallNotes() which loads all saved notes. Breakpoint is at save method in XmlNotebookrepo which is hit after creating Note in the UI.

Errors are temporary, giving up is permanent

So I have figured out what was wrong. I read about XmlAccessorTypes like Field,Property and Public and also XmlAdapters. Now I have my Note saved in Xml partially AngularObjects and GUI-config is remaining and loading of notes i.e unmarshalling.

note.xml