Running Batch Jobs on Amazon ECS -

-

i'm new using aws, , more ecs. currently, have developed application can take s3 link, download data link, processes data, , output information data. i've packaged application in docker container , resides on amazon container registry. want start cluster, send s3 link each ec2 instance running docker, have container instances crunch numbers, , return results single node. don't quite understand how supposed change application @ point. need make application running in docker container service? or should send commands containers via ssh? assuming far, how communicate cluster farm out work potentially hundreds of s3 links? ideally, since application compute intensive, i'd run 1 container per ec2 instance.

thanks!

your story hard answer since it's lot of questions without lot of research done.

my initial thought make stateless.

you're on right track making them start , process via s3. should expand use sqs queue. sqs messages contain s3 link. application start up, grab message sqs, process link got, , delete message.

the next thing not output console of kind. output somewhere else. different sqs queue, or somewhere.

this removes requirement boxes talk each other. speed things up, make infinitely scalable , remove strange hackery around making them communicate.

also why 1 container per instance? 2 threads @ 50% same 1 @ 100% usually. remove requirement , can use ecs + lambda + cloudwatch scale based on number of messages. >10000, scale up, kind of thing. <100 scale down. means can throw millions of messages sqs , let ecs scale process them , output somewhere else consume.


Comments

Post a Comment

Popular posts from this blog

java - Static nested class instance -

-
this question has answer here: java inner class , static nested class 23 answers i came across in java, know static nested class why create instance of 'static'. isn't whole idea of 'static' use without instance? understand concept of inner classes, why (and frankly how possible to) create 'instance' of 'static' member @ ? why this: 1- outerclass.staticnestedclass nestedobject = new outerclass.staticnestedclass(); 2- nestedobject.anestedclassmethod(); and not this: 1- outerclass outerinstance=new outerclass(); 2- outerinstance.staticnestedclass.anestedclassmethod(); use on inner classes, keyword static indicates can access inner class without instance of outer class. for example: public class outer { public static class inner { /* code here. */ } } with construction, can create instance of inn
Read more

c# - Bluetooth LE CanUpdate Characteristic property -

-
i building xamarin.forms cross platform mobile app, uses monkey.robotics bluetoth low energy functionality. connecting mbed based implimentation of custom gatt service . in xamarin c#, triggers characteristic. canupdate property in monkey.robotics? this standard example c# based on: if (characteristic.canupdate) { characteristic.valueupdated += (s, e) => { debug.writeline("characteristic.valueupdated"); device.begininvokeonmainthread( () => { updatedisplay(characteristic); }); isbusy = false; // spin until first result received }; isbusy = true; characteristic.startupdates(); } this has been working, since changed own custom gatt service connecting to, canupdate property false. property , how triggered? me debugging gatt service code. thanks monkey.robotics open sour
Read more

JavaScript - Replace variable from string in all occurrences -

-
ok, know if have character '-' , want remove in places in string javascript, ... someword = someword.replace(/-/g, ''); but, when applying array of characters, s not working ... const badchars = ('\/:*?"<>|').split(''); let filename = title.replace(/ /g, '-').tolocalelowercase(); (let item = 0; item < badchars.length; item++) { // below not work global '/ /g' filename = filename.replace(/badchars[item]/g, ''); } any ideas? /badchars[item]/g looks badchars , literally, followed i , t , e , or m . if you're trying use character badchars[item] , you'll need use regexp constructor, , you'll need escape regex-specific characters. escaping regular expression has been well-covered . using that: filename = filename.replace(new regexp(regexp.quote(badchars[item]), 'g'), ''); but , don't want that. want character class: let filename = title.replac
Read more