¥4.00元 Design data architecture for project integrating various structured datasets

发布于:2018-03-14 来自:freelancer 截止交稿时间截止交稿时间 2018-03-14 关注:72 评论:0

We have a science project that has outgrown its experimental setup and needs a new data architecture to enable further scaling. The task is to design one!

CURRENT SETUP WHICH NEEDS TO BE UPGRADED WITH A BETTER DATA ARCHITECTURE

We simply download the datasets from the structured data sources - in the native format that they are provided there (XML or .csv, for instance) - and store them locally.

Datafiles are then processed by R scripts, whereby one R script can be calling several locally stored datafiles, then processing them and storing the outputs locally again.

Different datasets can relate to one another with one or more common keys (identical variables).

Datasets range from several hundred to several hundred thousand observations.

SPECIFICATION

As you see, existing setup is rather primitive, and so it is to be replaced with a new data architecture. You are rather unconstrained in coming up with an optimal solution. However, you will not only need to make a proposal, but also justify your design to us (non-experts in the best practices for database management).

The task covers everything from

- the server choice: what local hardware or cloud service is appropriate for minimising the cost, yet still attaining full functionality?, to

- the type of the database able to efficiently handle datasets that will typically reach up to several GB in size, at most, to

- database update strategy that will allow to efficiently update our new database with new datafiles, where the source providers regularly - for instance, monthly - provide a new dump file containing an updated dataset. Likewise there should be an easy way to make an update from the source that provides an API, to

- ensuring and enabling full compatibility with and full optimisation for R programming language as a tool of choice to work with the data.

It is important that the new data architecture is future-proof: scalable and enabling multi-year projects that rely on the collected data.

BIDDING & CONTRACT

The job is in both articulating your proposed data architecture and in assisting with migration from the current setup.

Initially we request that you provide your:

- estimated fixed bid for the entire budget

- how many hours will it take you to complete most of the work

- your availability in hours per day.

The contractor will be selected based on the entire project bid. We understand that the specification will be further collaboratively refined during the project execution. Therefore please calculate your budget in such a way that it would cover most of the task described above (80%). Where we would like to have major extensions/additions, we will create a separate follow-up milestone with a separate budget.

Consequently this first project can possibly lead to an open-ended engagement on a milestone-based or hourly-rate retainer basis. We are therefore looking for a person who would be interested in/available for an extended collaboration.

In our experience with freelance contracting, we typically receive more qualified bids than we can award. Therefore, please have understanding that only shortlisted applicants will be contacted for the round two of additional questions & answers.

Thank you!

技能: Datatables, R 编程语言

查看更多: design data for various packings, architecture project brochure design, graphic design budget report architecture project, definition design data directory project, logo architecture project design company, data mining project design, design data entry project aspnet, design bbq school project, information system design data entry, data service project, data entry project based jobs, freelance data conversion project, data structure project, build data mining project, excel data mining project

免责声明:

任务易所有内容均为威客和外包行业网站提供或收集于互联网公开的信息,目的是给在网络上工作的威客和兼职人员收集更多的免费工作信息,以帮助更多的人自主就业。如果有内容触及您的权益,请给我们发邮件(kf@renwuyi.com)并附上具体网址和说明,核实后我们将立即删除!对免责声明的解释、修改及更新权均属于任务易所有。

你觉得这个任务肿么样?

我坚信,评论可以一针见血。
暂无评论
  • 评分:
    3.0分
  • Freelancer.com是世界上最大的自由职业、外包和众包市场,拥有许多用户和项目。我们将来自全球超过247个国家、地区的超过 9,471,792 雇主和 自由职业者联系在一起。通过我们的市场,雇主可以雇用自由职业者完成例如从软件开发、写作、数据输入和设计到 工厂、科学、销售、会计和法律服务的工作。

你可能也对这些任务感兴趣

  • 暂无数据

便捷搜索

价格
日内的任务