Call Scala & Spark code from Python -

-

i have codebase written in scala uses spark. i'd method foo called externally python code.

def foo(a: int, b: string): string

i saw here java python integration can use jython. think that's overkill though.

can't add pyspark method wraps scala/spark existing method?

if not, isn't there simpler solution, wouldn't need special module jython?


Comments

Post a Comment

Popular posts from this blog

php - Using grpc in Laravel, "Class 'Grpc\ChannelCredentials' not found." -

-
i trying use grpc in laravel project. this part of composer.json file: "require": { ... "datto/protobuf-php": "dev-master", "google/auth": "^0.7.0", "grpc/grpc": "dev-release-0_13", ... } i have generated php file proto file. placed in app/services/ , included in composer.json autoload.files section. have run composer install without issues. i have created route testing. has following code: $client = new vat_service\vatserviceclient('localhost:50051', [ 'credentials' => grpc\channelcredentials::createinsecure() ]); but receive response class 'grpc\channelcredentials' not found. when visit route. the laravel app running in homestead vagrant box. i don't know how proceed. missing dependency? did miss install something? i have solved problem, not sure of below steps solution. things tried: i installed grpc-beta in homestead box (...
Read more

github - Git errors while pushing -

-
Image
i input "git add -a" , gives me error: "fatal: unknown index entry format 65540000". i looked every other sources , not figure out means. then, tried pushing using sourcetree , got error below: all added codes , few imagefiles. not sure causing problem. last commit did month ago, btw. first commit after upgraded xcode 7.0 version. thanks this first commit after upgraded xcode 7.0 version then try clone again repo xcode7, , add change in new local repo. should work better in clone entirely managed new xcode.
Read more

java - Generating a pre-signed PUT url for Amazon S3 with a maximum content-length -

-
i'm trying generate pre-signed url client can use upload image specific s3 bucket. i've succesfully generated requests files, so: generatepresignedurlrequest urlrequest = new generatepresignedurlrequest(bucket, filename); urlrequest.setmethod(method); urlrequest.setexpiration(expiration); where expiration , method date , httpmethod objects respectively. now i'm trying create url allow users put file, can't figure out how set maximum content-length. did find information on post policies , i'd prefer use put here - i'd avoid constructing json, though doesn't seem possible. lastly, alternative answer way pass image upload api gateway lambda can upload lambda s3 after validating file type , size (which isn't ideal). while haven't managed limit file size on upload, ended creating lambda function activated on upload temporary bucket. function has signature below public static void checkupload(s3eventnotification event) { (this nota...
Read more