-  [JOIN IRC!]


[Return]
Posting mode: Reply
Name
Subject   (reply to 180)
Message
File
Password  (for post and file deletion)
¯\(°_O)/¯
  • Supported file types are: BMP, GIF, JPG, PNG
  • Maximum file size allowed is 10000 KB.
  • Images greater than 400x400 pixels will be thumbnailed.
  • Currently 317 unique user posts. View catalog

  • Blotter updated: 2015-09-02 Show/Hide Show All


File 133601626690.jpg - (1.68MB , 1920x1080 , 1332568510819.jpg )
180 No. 180
Hey, beautiful people, I'm having a hard time trying to even understand what I'm being asked for an assignment for university.
I'd rant for hours about this, but the gist of it is that we're supposed to make a system composing of:
- a file system client that is called by a linux VFS through FUSE
- a cache implemented with memcached
- a remote file system in ext2
The idea is to be able to manage files from the client machine, that will look it up in the cache. If it isn't there, it'll make a request to the RFS.
The problem is, I don't even know how to begin. I was told that we could start by writing functions that read superblocks, inodes, etc, but I don't even know how the fuck I'm supposed to do that. We're working with a ext2 file acting like a drive formatted in ext2, and I don't know if I should map it into memory, use unlocked stream operations (I'm not even really sure what that is about though, and there doesn't seem to be much on it on the web).
So yeah, any help at all would be extremely appreciated. Sorry if it sounds a bit convoluted, I'm just really frustrated right now.
>> No. 181
I'm a little confused by your description. Maybe terminology is part of your difficulty identifying the problem (that can make a task feel really overwhelming).

I had written a bit of blather about your three point definition, because alone it doesn't really make sense. But the first sentence after that clears things up enough.

You need to make some arbitrary decisions, primarily about what will be kept in the cache and what will you purge. Memcached has a lot of easy ways to make decisions like this, so read up on it (and you'll feel better already). Memcached also has mechanisms to identify what it doesn't currently hold and make a call to whatever you define as the primary resource to fetch it.

So your real problem is just creating a VFS -- as in, an abstraction over the real filesystem(s) that pretends to be a real filesystem of its own complete with file system type calls for writing, retrieval, deletion, etc. You can do whatever you want, but I recommend not getting fancy -- just write a wrapper over normal filesystem functions that are already defined, and based on where the resource actually is (which requires mapping, which is your *real* task here) translates the fetch (or whatever) request into the real FS's expected call and back again.

Let me stress, you are writing a filesystem wrapper which utilizes a local cache which is already implemented by memcached (once you learn how to use it), not an independent caching mechanism or anything really crazy. Decide how your interface is going to mount/access the filesystem you're using and how it will present that filesystem to userland -- and that's about it. FUSE and memcached already have the hard parts taken care of.

As far as mapping stuff into memory, that's what memcached is for. You've got some reading and playing to do, but this isn't that hard and there are many fine examples of excellent implementation out there already. Most of them do stuff you don't need to worry about, like security implementations, versioning, replication, striping/network-fragmentation, etc. but reading a bit about their design at a higher level might be informative (AFS, Globus, GFS, NFS, etc).

blah blah. This post is a bit scattered. The docs for memcached and fuse are too (sorry... lulz). But you can get this done if you familiarize yourself with these tools. Anyway, play with this and have fun. This is the beginning of learning how really cool stuff works, so have fun, not stress.
>> No. 185
>>181
Thanks! And yeah, I'm sorry. English is not my native language, and translating technical terms is kind of hard. But yeah, I've been playing a bit with systemcalls in C to see how I can work with a ext2.disk file. It's not exactly rocket science, I know, but I have three months to develop it, and so far we've seen mostly object-oriented languages in class.
So seriously, thank you a lot! I've read up on FUSE and Memcached a bit already, but what I was mostly concerned about was the VFS part. Now I have an idea about how to go on with it.


Delete post []
Password  
Report post
Reason