Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
C
CodiMD export script
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
stgroup
misc
CodiMD export script
Commits
8f1987dc
Commit
8f1987dc
authored
2 years ago
by
Andreas Domanowski
Browse files
Options
Downloads
Patches
Plain Diff
Add instructions on importing to readme
parent
e2f11888
No related branches found
No related tags found
1 merge request
!1
Hedgedoc import
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
README.md
+53
-30
53 additions, 30 deletions
README.md
with
53 additions
and
30 deletions
README.md
+
53
−
30
View file @
8f1987dc
# CodiMD
note ex
porter
# CodiMD
Note Exporter + Hedgedoc Note Im
porter
This little tool is intended to help you backup the notes that you have in your CodiMD history to a
local folder. It complements CodiMD's "export user data" functionality, which only downloads
documents that you own.
This little tool is intended to help you
**backup**
the notes that your CodiMD history shows to a local folder.
It complements CodiMD's "export user data" functionality, which only downloads
documents that you are the owner of.
This backup is not meant to be uploaded by you in its entirety, since this would create ownership issues of shared
documents.
The export user data functionality of CodiMD creates an archive containing all the documents you are the owner of.
This tool furthermore implements functionality to upload each or a selection of your owned documents based on this
archive.
Based upon the path the document was served under at CodiMD, this script tries to migrate the document to the exact same
path if possible, e.g., https://md.inf.tu-dresden.de/my_custom_path gets mapped
to https://md.inf.tu-dresden.de/notes/my_custom_path
Be aware that this approach is a little hacky.
Unfortunately, this is necessary since our CodiMD instance did use KeyCloak as an OAuth provider.
Follow the steps closely to ensure that everything works as intended and that the ownership of documents is not messed
up.
## Requirements
...
...
@@ -10,9 +25,21 @@ Its only dependency is Python >= 3.7 (use `python3 -V` to check your version).
## Usage
### 1. Extract value of the session cookie
After cloning this repository, follow the following steps to migrate your documents.
If you only want to export your visited documents and import them manually, just follow steps 2 and 4.
### 1. Export Your Owned Documents From Our Codimd Instance and Weed Them Out
-
Go to https://md.inf.tu-dresden.de/
-
Click on your username in the upper right and hit "
*Export user data*
"
-
Download the file "archive.zip" and place it in the same folder as this file (README.md)
-
**Weed the archive out**
! Most likely, it will contain a lot of unused or empty documents. Delete those files in place
in the archive, don't extract them.
First of all, you need to extract the value of the
`connect.sid`
browser cookie for the
### 2. Extract the value of your CodiMD session cookie
To ensure that the script can access the documents you accessed in our CodiMD instance, you need to extract the value of
the
`connect.sid`
browser cookie for the
particular CodiMD instance that you use (e.g.,
`md.inf.tu-dresden.de`
).
This procedure is necessary because the CodiMD login is handled externally by Keycloak.
...
...
@@ -22,38 +49,34 @@ password**. Don't share it with others!
Now, the instructions are similar for Chrome and Firefox:
1.
Navigate your browser to the CodiMD instance
.
2.
Open developer tools.
1.
Navigate your browser to the
[
CodiMD instance
](
https://md.inf.tu-dresden.de/
)
. Be sure that you are logged in
2.
Open developer tools
(F12)
.
3.
**Chrome**
: go to the "Application" tab.
**Firefox**
: go to the "Storage" tab ("Web-Speicher").
4.
Un-collapse "Cookies".
5.
In the list, search for a cookie with the name
`connect.sid`
6.
Select and copy the value. It must start with the character sequence
`s%3A`
.
### 3. Extract the value of your HedgeDoc session cookie
Follow the instructions from step 2, but copy the value of the cookie key
`connect.hedgeDoc.sid`
###
2
. Execute the script via the command line
###
4
. Execute the script via the command line
Clone this repository to a path of your liking and just execute
`./codimd_
export.py`
in a
shell. The script will ask for your session id and download the notes to the relative path
Execute
`./md-import-export.py`
or
`python3 md-import-
export.py`
in a
shell. The script will ask for your session id
s
and download the notes
you visited
to the relative path
`./codimd-documents`
.
Please note that your CodiMD history might reference already deleted notes or notes you no
longer have access to. The URLs of these inaccessible notes are listed as part of the
script's output.
### Example
$️ ./codimd_export.py
Please provide your CodiMD session id (connect.sid cookie):
HTTP 404 Not Found: https://md.inf.tu-dresden.de/TOPofLSB
HTTP 404 Not Found: https://md.inf.tu-dresden.de/6jeKGnJnSp6H0f0mVnBwQQ
HTTP 403 Forbidden: https://md.inf.tu-dresden.de/klausur-2020-experiences
HTTP 404 Not Found: https://md.inf.tu-dresden.de/iQVB7TO-QXqi3hGSZ7J4Eg
HTTP 404 Not Found: https://md.inf.tu-dresden.de/LSB-2020-11-17
HTTP 404 Not Found: https://md.inf.tu-dresden.de/Nzbj13ydRLe1d0fRCyG6cA
HTTP 404 Not Found: https://md.inf.tu-dresden.de/LSB-2021-02-02
HTTP 404 Not Found: https://md.inf.tu-dresden.de/LSB-2021-06-22
HTTP 404 Not Found: https://md.inf.tu-dresden.de/LSB-2021-12-07
HTTP 404 Not Found: https://md.inf.tu-dresden.de/ThesisTemplateBachelor
HTTP 404 Not Found: https://md.inf.tu-dresden.de/cnoGl7hoTaK1JIh2-u-h0g
Done: 163 notes successfully downloaded, 11 not accessible.
output of the script.
Furthermore, the script tries to upload every file with a
`.md`
extension to our HedgeDoc instance.
### 5. Visit the Uploaded Documents to Make Them Appear in Your Hedgedoc History
Uploading the documents is not enough to make them appear in your HedgeDoc history.
You need to visit them at least once to make them available.
The script automatically generates a file
`history_scripts/hedgedocu_documents_to_visit.url`
.
You can either visit every file manually or execute the bash script
`history_scripts/visit_migrated_documents.sh`
.
This script will ask you for a browser to open the documents and then open them in batches of 50 URLs.
Be aware: this might be pretty resource intensive and might take a while.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment