Bulk Upload csv to Neo4j on windows

This blog post was originally intended only for data based philosophical discussions. However, given that I have not posted anything in many a years I decided to dilute it.

I had lot of problems bulk importing csv data in to neo4j. I have finally figured out how to do it.

neo4j is a graph db. I won't go in to details of what is a graph db or how to write cypher queries.
once you have downloaded neo4j from  here and installed neo4j you will see the neo4j desktop app.

Add a new graph. create a new graph give it a password. once the graph is created click on manage and go to the "Terminal" tab.






Now before we go ahead you need to prepare your csv files. your csv files should be split as node csv and relations svs. And the initial headings have to follow a syntax.
In your node csv the first column should be unique key and your first column heading should be
<your id>:ID(<your node name>)

In your relations csv you must have start and end nodes. (note that they can be different nodes eg. user and property.)
you can chose to ignore other columns in these csvs if they are not ignored they are captured as attributes. (see https://neo4j.com/developer/guide-import-csv/ for more details)

bin\neo4j-admin import --mode=csv --nodes:User=import\<your node csv> --relationships:FOLLOWS=import\<your relationship csv> --ignore-missing-nodes=true --multiline-fields=true

eg:
bin\neo4j-admin import --mode=csv --nodes:User=import\user.csv --relationships:FOLLOWS=import\User_FOLLOWS.csv --ignore-missing-nodes=true --multiline-fields=true

Note the only difference between windows and linux is / and \ . Most of the documentation you find online will be for linux. so when the document asks you to use "ls" use "dir" instead.

once you have run this command. be sure to hit the play button on top before you connect to the database. you can bulk import csv in to a neo4j db only once. It does not create transaction logs and it is meant for one time first upload only.

If you want to try this with a small file you can use the csvs in my Git page.
Download sample csvs place them in the correct import folder and run this command in the neo4j terminal.
bin\neo4j-admin import --mode=csv --nodes:Worker=import\id.csv --relationships:FOLLOWS=import\relations.csv --ignore-missing-nodes=true --multiline-fields=true

The full video is on youtube


Comments

Post a Comment

Popular posts from this blog

Virus: Discussing macroscopic possibilities

Image
Our world view is limited by our own senses and our size and scale. I am throwing out a possibility of studying viruses through another lens.  Many people believe that COVID-19 is the mother earth’s fight against human infection [ 0 ] . It is quite possible that viruses are earth’s antibodies trying to keep the earth healthy. Viruses are known to keep the balance and diversity of microbes in the oceans.   [1]   Viruses have been eliminating species that run wild since the beginning of life on earth. They are a major driver in human evolution [   2 ][   3 ] . Recent studies have shown that viruses could have been the reason we have life on earth. [   4 ][   5 ] So, there is more to viruses than it is commonly known. In this blog, I would like to discuss the possibility of studying viruses through another lens. Allow me to take you through some seemingly unrelated illustrations to make my point.   Every mammal on this planet instinctively develops a natural equilibrium with the surroundi
Read more

Could Generative AI pose an existential threat to us?

Image
  In recent days, prominent figures in the field of AI, like Geoffrey Hinton, Sam Altman, etc., have sparked discussions about the potential dangers of generative AI.  There is something eerie about chat GPT-like bots. One can think of many dangers with the current AI tech, but none of them seem existential.  Andrew NG  called for the LinkedIn community to  list the existential risks of AI . While it is challenging to pinpoint the exact reasons behind these concerns, there is an undeniable discomfort associated with the latest advancements in AI technology. This article is an attempt to evaluate fears surrounding generative AI, examining its impact on various aspects of our lives and contemplating whether it poses a terminal threat to the human species. 1. The Misdemeanor Bots: When interacting with chat GPT-like bots, one cannot help but think of less dangerous spam emails or encountering age-inappropriate content. Although these experiences might not be existential threats, they high
Read more

Advantage; Homeloan

Image
Get a home loan save tax. Yes, I'll elaborate on it later but now some quick notes Get a home loan and if you live in it you get up to 1.5Lakh of interest deducted from your taxable income Get a home loan and rent it out; all the interest is delectable form your taxable income. Get your spouse to be a co-applicant on the loan both of you can enjoy the deductions. Up to 1 Lakh of Principle is delectable  from your taxable income (this falls under 80C so your PPF, ULIP, MF etc. is wasted) (Rented out or self-occupied) Use the interactive Tableau dashboard (change the numbers in the boxes) to see how the numbers work for you and what saves most taxes for you. Learn About Tableau You will notice that you will have better tax savings when you rent out your property and stay in a rented house yourselves. when you rent out all your Interest is tax deductible) But if you stay in it yourselves - you only get up to Rs 1.5 lakhs of deductible amount per year. [indirect me
Read more