Syncing local external storage: work-around

Ask all your questions regarding OC 7.x Please read the Support Forum Rules
Forum rules
ownCloud 7.x will be end of life soon. See maintenance and release schedule for more details.

Time to upgrade!

Before you post; make sure you are using at least PHP Version 5.3.x - Also read Support Forum - Read this before posting
sm8ps
Newbie
Posts: 5
Joined: Sun Nov 23, 2014 10:36 pm

Syncing local external storage: work-around

Postby sm8ps » Tue Jun 30, 2015 12:53 am

According to the discussion on Github about several issues with external storage not being synced, I wanted to share a work-around that I discovered after quite some fruitless searching. The principal idea is not mine but I publish it here in the hope that it may save others some time.

Obviously the server does not communicate changes in (local) external storage to the client, although the files are updated in the web-interface. It is possible to force a full file scan on the server as a cron-job but in my case that takes around five minutes each and thus is prohibitive.

The trick now lies in issuing a sync from the client side. For that it is enough to regularly touch a dummy file every few minutes as a cron-job. I have not given much thought to how much traffic this will generate each time but AFAIKT it seems the load on both server and client is very modest. This is on Ubuntu 12.04 server and client with OC7 and the current 1.8.2 client.

To be very specific, here is a full description:
. Server with (local) external storage 'serverExtStorage/'
. Client with local directory 'myExtStorage/' synced with 'serverExtStorage/'
. empty dummy file 'serverExtStorage/.touchMe'
. crontab for user: "*/4 * * * * touch <PATH>serverExtStorage/.touchMe"

This now gives a perfect combo of OwnCloud and SyncThing for keeping data on Android devices in sync with OwnCloud. I cannot say if this also works with non-local external storage but I whould certainly guess so.

Cheers!

tflidd
Forum Moderator
Posts: 7159
Joined: Sat Dec 07, 2013 7:27 pm
ownCloud version: 8.2.3
Webserver: Apache
Database: MySQL
OS: Linux

Re: Syncing local external storage: work-around

Postby tflidd » Tue Jun 30, 2015 8:43 am

Thanks for sharing this workaround. The central topic on github is here https://github.com/owncloud/core/issues/11797, solution in scheduled for OC 8.2.

gavin.kennedy
Beginner
Posts: 12
Joined: Mon Oct 13, 2014 9:45 am
ownCloud version: 7.0.4
Webserver: Apache
Database: MySQL
OS: Linux

Re: Syncing local external storage: work-around

Postby gavin.kennedy » Fri Jul 03, 2015 6:11 am

Great timing - I've just been working through a solution to test if our owncloud clients are syncing.

Because ownCloud client doesn't log by default, we have no way of knowing the status of the client.

We run all of our users on external storage, which are SMB shares, and what I've done is created a file on each user's home folders on the external storage using a script. Inside this file I write the date and time that the script runs (every 10 minutes).

I noticed though, that the file I was creating was not being downloaded by the client onto the machines (this is how I was going to tell if the sync was working), despite it being modified (new date/time written to file) every 10 minutes. ownCloud does not recognize the file as being modified on the external storage, so therefore does nothing. This is somewhat understandable, as without doing a complete scan of the external storage, how is it supposed to know? I did read something about 8.2 addressing this somehow, although because I'm short of time I didn't dive too deep into it.

This workaround that you've posted, by touching a file in the same folder using the client, has prompted ownCloud to download the modified file on the external storage. Working well, currently.

Thanks so much for posting your work-around - it's a very simple solution that's working well for us.

Cheers,
Gav

sm8ps
Newbie
Posts: 5
Joined: Sun Nov 23, 2014 10:36 pm

Re: Syncing local external storage: work-around

Postby sm8ps » Sat Jul 04, 2015 10:16 am

gavin.kennedy wrote:We run all of our users on external storage, which are SMB shares
(...)
This workaround that you've posted, by touching a file in the same folder using the client, has prompted ownCloud to download the modified file on the external storage. Working well, currently.

Glad my work-around is useful to you! Your post also shows that it works with non-local external storage as well.

sm8ps
Newbie
Posts: 5
Joined: Sun Nov 23, 2014 10:36 pm

Re: Syncing local external storage: work-around

Postby sm8ps » Thu Jul 09, 2015 12:28 am

I may have spoken too soon because in the meantime my work-around has not proven as successful as upon first try. It seems that the touching of a file does not trigger a *recursive* scan of the directory. So I have resorted to a more brute version, namely touching a dummy file in each and every sub-directory of the external storage. The following entry in the personal crontab seems to work as of now:

Code: Select all

*/4 * * * * for i in $(find ~<PATH>/myExtStorage/ -type d) ; do touch "$i"/.touchMe ; done

(BTW: I had messed up the original version of this by erroneously putting in "serverExtStorage" instead of "myExtStorage".) For the moment this seems to work as expected (or better: hoped for :) ).

lifeboy
Beginner
Posts: 18
Joined: Mon Jul 13, 2015 2:12 pm
ownCloud version: 9.0.1
Webserver: Apache
Database: PostgreSQL
OS: Linux
PHP version: 7.0
Location: ZA
Contact:

Re: Syncing local external storage: work-around

Postby lifeboy » Mon Jun 13, 2016 2:21 pm

sm8ps wrote:I may have spoken too soon because in the meantime my work-around has not proven as successful as upon first try. It seems that the touching of a file does not trigger a *recursive* scan of the directory. So I have resorted to a more brute version, namely touching a dummy file in each and every sub-directory of the external storage. The following entry in the personal crontab seems to work as of now:

Code: Select all

*/4 * * * * for i in $(find ~<PATH>/myExtStorage/ -type d) ; do touch "$i"/.touchMe ; done

(BTW: I had messed up the original version of this by erroneously putting in "serverExtStorage" instead of "myExtStorage".) For the moment this seems to work as expected (or better: hoped for :) ).


This doesn't work when you have paths or filenames with spaces. Therefor a safer way would be to:

Code: Select all

find <PATH>serverExtStorage/ -type d -exec touch {}/.touchme ';'


the "-exec" option does the trick to preserve the spaces, rather that seeing them as separators.


  • Similar Topics
    Replies
    Views
    Last post

Return to “ownCloud Community Edition 7.x”

Who is online

Users browsing this forum: No registered users and 3 guests