OwnHack: Import ICS to OwnCloud via cronjob

Tutorials on ownCloud administration
Forum rules
The forums were migrated over to https://central.owncloud.org which is based on the forum software Discourse. The forums here is put into read-only mode starting from today.

More background information about this move and the reasoning behind it is available in this blogpost:

https://daniel.molkentin.net/2016/07/20 ... d-central/
Arney
Newbie
Posts: 5
Joined: Thu May 30, 2013 2:04 am
ownCloud version: 5.0.6
Webserver: Apache
Database: MySQL
OS: Linux

OwnHack: Import ICS to OwnCloud via cronjob

Postby Arney » Thu May 30, 2013 2:17 am

Hi there,

I was looking for a way to keep my OwnCloud calendar in sync with some online ICS files. Not too easy, but here's the hack:


What the following script will do:
  • download the ICS files from the web
  • chop it up into one file per VEVENT
  • upload each file to the calendar using cadaver

And here is the script:

Code: Select all

#!/bin/bash

# What to import
calendars=(http://www.ifeiertage.de/by-s.ics http://bildungsklick.de/schulferien/bayern/ical)

# Ensure an empty temporary subdirectory
ScriptDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

for calendar in ${calendars[@]}
do
  wget $calendar -P $ScriptDir
  file=${calendar##*/}

  bevents=($(grep -n BEGIN:VEVENT $file | cut -d: -f1))
  eevents=($(grep -n END:VEVENT $file | cut -d: -f1))

  ehead=${bevents[0]}
  let "ehead -= 1"

  bfoot=$(wc -l < $file)
  let "bfoot -= ${eevents[-1]}"

  for (( i=0; i < ${#bevents[@]}; i++))
  do
    head -n $ehead $file > $file-$i.ics
    sed -n ${bevents[$i]},${eevents[$i]}p $file >> $file-$i.ics
    tail -n $bfoot $file >> $file-$i.ics
  done

  rm $file
done

cadaver https://owncloud.domain.com/remote.php/caldav/calendars/username/syncedcalendar << EOC
  mput *.ics
  exit
EOC

rm -r *.ics


Make it a cronjob and the fun is yours!

karlito139
Newbie
Posts: 3
Joined: Sun Sep 15, 2013 3:38 pm

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby karlito139 » Sun Sep 15, 2013 3:43 pm

Hi,


That's awesome. Thanks.

I got some difficulties with my certificate. I have a self signed certificate and each time I try to log using cadaver it says my certificate is not trusted and ask me if I'd like to trust this certificate. You refer to this site : http://www.mavetju.org/unix/netrc.php to resolve this problem, but I can't see any solutions.

RealRancor
ownCloud master
Posts: 17381
Joined: Sat May 26, 2012 3:00 pm
ownCloud version: 9.0.2
Webserver: nginx
Database: MySQL
OS: Linux
PHP version: 7.0.x

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby RealRancor » Sun Sep 15, 2013 4:03 pm

Hi,

one of the first hits on google when searching for:

cadaver self signed certificate

leads you to this thread:

http://stezz.blogspot.de/2007/08/webdav ... cates.html

where you could find a workaround for this problem.
*gone*

aj-network
Newbie
Posts: 4
Joined: Thu Aug 15, 2013 3:32 pm
ownCloud version: 5.0.10
Webserver: Apache
Database: MySQL
OS: Linux
PHP version: 5.3.3

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby aj-network » Wed Oct 02, 2013 2:56 pm

Hi,

I have tried your scipt but getting different errors. Here the output of the Script:
server@AJ-Server:/mnt/500gb/Terminplan$ ./ocUpdateCal.sh
--2013-10-02 14:52:49-- http://aj-network.dyndns.org/Uni/basic.ics
Auflösen des Hostnamen aj-network.dyndns.org... 192.168.1.2
Verbindungsaufbau zu aj-network.dyndns.org|192.168.1.2|:80... verbunden.
HTTP-Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 24789 (24K) [text/calendar]
In »/mnt/500gb/Terminplan/basic.ics« speichern.

100%[======================================>] 24.789 --.-K/s in 0,02s

2013-10-02 14:52:49 (1,11 MB/s) - »/mnt/500gb/Terminplan/basic.ics« gespeichert [24789/24789]

./ocUpdateCal.sh: Zeile 21: eevents: Falscher Feldbezeichner.
./ocUpdateCal.sh: Zeile 21: let: bfoot -= : Syntax Fehler: Operator erwartet. (Fehlerverursachendes Zeichen ist \"-= \").
./ocUpdateCal.sh: Zeile 38: Warnung: here-document at line 33 delimited by end-of-file (wanted `EOC')
dav:/owncloud/remote.php/caldav/calendars/Alexander/test/> mput *.ics
[Matching... 64 matches.]
Uploading basic.ics-25.ics to `/owncloud/remote.php/caldav/calendars/Alexander/test/basic.ics-25.ics':
Progress: [=============================>] 100,0% of 25310 bytes failed:
415 Unsupported Media Type
.......
Uploading basic.ics-11.ics to `/owncloud/remote.php/caldav/calendars/Alexander/test/basic.ics-11.ics':
Progress: [=============================>] 100,0% of 25301 bytes failed:
415 Unsupported Media Type
dav:/owncloud/remote.php/caldav/calendars/Alexander/test/> exit
Connection to `aj-network.dyndns.org' closed.


And here my Config-File:

Code: Select all

    #!/bin/bash

    # What to import
    calendars=(http://aj-network.dyndns.org/Uni/basic.ics)

    # Ensure an empty temporary subdirectory
    ScriptDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

    for calendar in ${calendars[@]}
    do
      wget $calendar -P $ScriptDir
      file=${calendar##*/}

      bevents=($(grep -n BEGIN:VEVENT $file | cut -d: -f1))
      eevents=($(grep -n END:VEVENT $file | cut -d: -f1))

      ehead=${bevents[0]}
      let "ehead -= 1"

      bfoot=$(wc -l < $file)
      let "bfoot -= ${eevents[-1]}"

      for (( i=0; i < ${#bevents[@]}; i++))
      do
        head -n $ehead $file > $file-$i.ics
        sed -n ${bevents[$i]},${eevents[$i]}p $file >> $file-$i.ics
        tail -n $bfoot $file >> $file-$i.ics
      done

      rm $file
    done

    cadaver http://aj-network.dyndns.org/owncloud/remote.php/caldav/calendars/Alexander/test << EOC
      mput *.ics
      exit
    EOC

    rm -r *.ics



It seems like it is not creating the .ics files correctly. There is always the complete input-ics-File in the sub-Files.

Any possibilities?

Greet aj-network

makaber
Newbie
Posts: 1
Joined: Tue Jul 22, 2014 5:22 pm
Webserver: Apache
Database: MySQL
OS: Linux
PHP version: 5.3

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby makaber » Tue Jul 22, 2014 6:04 pm

time to bring up discussion again. :-)
i´ve got a lot of courses (university) here. i can export a single ics-file for each course.
the script above is working well. but it imports the ics to a single calendar in a owncloud account.
my idea is:
dynamic creation of multiple calendars in one single account by using cadaver. one calandar per course.
the problem is: cadaver cannot create calendars in owncloud. i get an error 403.

Code: Select all

/var/calendar# cadaver -t https://cloud.anyhost.de/remote.php/caldav/calendars/stundenplan
dav:/remote.php/caldav/calendars/stundenplan/> mkcol course1
Creating `course1': failed:
403 Forbidden
dav:/remote.php/caldav/calendars/stundenplan/>

"stundenplan" is the name of the account/user
course1 should be an example for the name of a single calendar.
any ideas?
i´m using oc7 rc3

wolfgang_k
Beginner
Posts: 41
Joined: Thu Jan 02, 2014 6:35 pm

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby wolfgang_k » Fri Sep 12, 2014 11:12 am

Hello!

I followed all the links provided here and I've been trying for hours now, but I can't get cadaver to accept my self signed certificate that I use in my OC. I even recompiled the source code as said here:

http://stezz.blogspot.de/2007/08/webdav ... cates.html

All in vain. Each time I run cadaver I am asked if I want to trust the certificate. I am out of ideas. Cadaver is the only program now that makes trouble, all other programs accept the certificate without any mocking.

What else can I do to make cadaver permanently accept this certificate?

wazzle638
Newbie
Posts: 1
Joined: Tue Oct 28, 2014 12:39 pm

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby wazzle638 » Tue Oct 28, 2014 12:49 pm

Thanks for your awesome script.

I build something more out of it, using search results from allover internet and patching it together. And it uses curl, rather than cadaver, so it's a bit more portable.

The script I post here does the following:
- it queries for all calendars of a user
- reads the "calendar-description" of each calendar
- takes that as the url of a remote calendar
- fetches the remote calendar
- does the splitting using your script
- uploads the ics files to the calendar whose description gave the remote url.

This means that even slightly less intelligent users can use this method, by adding this user to their calendar programme, and add a calendar with as a description strictly the url of the remote calendar. No need for admin's adding calendars etc. etc.

For safety, I do create a separate user which has all the harvested calendars, just for safety, such that no real users calendar gets screwed if the script goes crazy AND because this script stores that password in plain text. My user is Harvest, and it also has a simple password. The script is written to be run on the server that hosts your owncloud. Also beware, my urls actually correspond to the Baikal server (because it is way faster if you only care about caldav/carddav), but the backend is Sabredav, so identical to OwnCloud.

Code: Select all

#!/bin/bash

# settings:
# e.g. HOSTNAME=owncloud.myserver.com
HOSTNAME=PUTYOURHOSTNAMEHERE
USER=PUTYOURHARVRESTINGUSERNAMEHERE
PASSWORD=PUTYOURPLAINPASSWORDHERE
# end settings

TmpDir=`mktemp -d /tmp/TMP4HarvestXXX`

# get the calendars
curl -s -k --digest  -u "${USER}":"${PASSWORD}" -H "Host: ${HOSTNAME}" -X PROPFIND  https://127.0.0.1/cal.php/calendars/"${USER}" -d '<d:propfind xmlns:d="DAV:" xmlns:cs="http://calendarserver.org/ns/"  xmlns:C="urn:ietf:params:xml:ns:caldav">
  <d:prop>
     <d:displayname /> <C:calendar-description xml:lang="en"/>
  </d:prop>
</d:propfind>' | sed 's,</d:response>,</d:response>\
,g' | grep -i -v "404 Not Found" | grep -i d:response | while read -r line; do

    url=`echo $line | sed -e 's,.*<d:href>\([^<]*\)</d:href>.*,\1,g' -e 's,\&amp\;,\&,g'`
    url=`echo $url | tr -d '\n' | tr -d '\r'`
    remoteurl=`echo $line | sed -e 's,.*<cal:calendar-description>\([^<]*\)</cal:calendar-description>.*,\1,g' -e 's,\&amp\;,\&,g'`
    remoteurl=`echo $remoteurl | tr -d '\n' | tr -d '\r'`
    relurl=`echo $url | sed 's,.*/\([^/]*\)/$,\1,g' | tr -d '\n' | tr -d '\r'`

    echo "URL: " $url
    echo "REMOTE URL: " $remoteurl
    echo "REL URL: " $relurl

    fname="${TmpDir}"/"${relurl}".ics
    curl -s -k "${remoteurl}" > "${fname}"
    # end with newline:
    echo >> "${fname}"
    file="${fname}"

    # skip empty calendars:
    if ! grep "BEGIN:VEVENT" "${file}" > /dev/null; then continue; fi

    bevents=($(grep -n BEGIN:VEVENT "$file" | cut -d: -f1))
    eevents=($(grep -n END:VEVENT "$file" | cut -d: -f1))

    ehead=${bevents[0]}
    let "ehead -= 1"

    bfoot=$(wc -l < $file)
    let "bfoot -= ${eevents[-1]}"

    for (( i=0; i < ${#bevents[@]}; i++))
    do
        head -n $ehead $file > $file-$i.ics
        sed -n ${bevents[$i]},${eevents[$i]}p $file >> $file-$i.ics
        tail -n $bfoot $file >> $file-$i.ics
    done

    # put the individual events, use UID as filename.
    for (( i=0; i < ${#bevents[@]}; i++))
    do
        uid=`grep -e "^UID:"  $file-$i.ics | sed 's/UID://' | tr -d '\n' | tr -d '\r'` 
        # write "echo" in front of curl if you only want to test.
        curl -s -k --digest -u "${USER}":"${PASSWORD}" -H "Host: ${HOSTNAME}" -T $file-$i.ics  https://127.0.0.1"${url}${uid}".ics
    done
done

rm -rf "${TmpDir}"



Forgive me the ugly coding style. But it's a patch, and it does the job.

Mathieu Denat
Newbie
Posts: 1
Joined: Thu Dec 11, 2014 3:06 pm
ownCloud version: 7.0.3
Webserver: Apache
Database: MySQL
OS: Linux

Re: OwnHack: Import ICS to OwnCloud via cronjob

Postby Mathieu Denat » Thu Dec 11, 2014 3:40 pm

Hi all, (I'm French, sorry for bad english)

Thanks for the script.

I'm hosting my own OwnCloud server at home, and the certification to use SSL was laborious.
I've encountered some problems, so I submit you few funded solutions:

1. My certificate wasn't accepted by the owncloud client (for Ubuntu) because I use self-signed certificate which doesn't indicate the common name of website (Common Name (e.G. server FQDN or YOUR name).
So syncing was interrupted. To resolve problem, I found a french tutorial (sorry!) wich permit to certificate myself my own certificat:
http://www.linux-france.org/prj/edu/arc ... 24s03.html
It explain how to become a "Certfication Organisation"! Following howto permit to resolve problem.

2.I use an self-hosted server, so I needn't use FTP, so to permit to cadaver to connect to the server, I use .netrc with following syntax:

Code: Select all

machine my.host
          login my owncloud login
          password my owncloud password

And it's running without ftp connection.

Thanks again for the script.
Have nice day.


  • Similar Topics
    Replies
    Views
    Last post

Return to “Tutorials”

Who is online

Users browsing this forum: No registered users and 2 guests